Three-dimensional image communication terminal and projection-type three-dimensional image display apparatus

ABSTRACT

It is possible to provide a three-dimensional image display apparatus which can reproduce a three-dimensional image having an appearance of solidity with a simple configuration. The three-dimensional image reproduction apparatus includes: a display device  2  for displaying two-dimensional element images having a plurality of horizontal and vertical parallax images  1  containing information on a parallax and an image at the time of reproducing a three-dimensional solid image  7 ; and a lens  3  for forming the three-dimensional solid image  7  at a predetermined spatial intersection from the element images displayed by the display device  2 . Here, the lens  3  diffracts the element images and forms the three-dimensional image by the use of a diffraction effect when the element images are emitted from the display device  2.

FIELD OF THE INVENTION

The present invention relates to a three-dimensional image displayapparatus capable of three-dimensionally displaying an image, and moreparticularly, to a projection-type three-dimensional image displayapparatus capable of three-dimensionally displaying an image with anexcellent realistic feeling, which is widely used in the fields of imagetechnology, amusement, entertainment, Internet, information, multimedia,communication, advertisement and promotion, medicine, art, education,design support, simulation, virtual reality, and the like.

BACKGROUND INFORMATION

Conventionally, as means for displaying a three-dimensional image basedon information on the three-dimensional image and allowing an observerto recognize the three-dimensional image, there were known a naked-eyestereoscopic paralleling method in which a right image of two imageshaving a binocular parallax is viewed with a right eye and a left imageis viewed with a left eye, a stereoscope method in which an image isviewed using a pair of spectacles fitted with a liquid shutter or usingdifferent lenses for the right eye and the left eye, an anaglyph methodin which a binocular parallax image having different colors of red andblue is viewed with red-blue spectacles. However, wearing specialspectacles or training an observer is required for the observer to viewa three-dimensional image using the above-mentioned methods.

In recent years, with development of a liquid crystal technology, liquidcrystal monitors capable of displaying an image without using specialspectacles were introduced to the market. Most of the liquid crystalmonitors are three-dimensional liquid crystal display apparatuses of animage splitter type without spectacles, that is, three-dimensional imagedisplay apparatuses of a so-called parallax barrier type or a lenticularlens type having only a horizontal parallax.

In the three-dimensional image display apparatuses of a parallax barriertype or a lenticular lens type, the appearance of solidity is created byspatially forming optical image paths so that a right-eye image isviewed at a right-eye position and a left-eye image is viewed at aleft-eye position. Accordingly, since the optical image path isspatially and periodically formed at the right-eye position and theleft-eye position, the appearance of solidity is deteriorated when theoptical image paths depart from the fixed positions. In addition, sinceimages having a horizontal parallax are formed in principle, theappearance of solidity is deteriorated when the right-eye position andthe left-eye position depart from the horizontal direction. Therefore,when it is intended to carry out stereoscopic views while maintainingthe appearance of solidity of a three-dimensional video for a long time,it is necessary to fix the right-eye position and the left-eye positionto predetermined positions in space, respectively.

As for the horizontal dislocation of the right-eye position and theleft-eye position, there has been suggested a method of specifyingpositions of an observer's eyes or a position of the observer's facewith a sensor and controlling and correcting the optical image path inaccordance with the dislocation of the specified position. However, inthis case, there is a problem that the apparatus increases in size andthus markers should be attached to the observer so as to sense thepositions of the eyes or the position of the face.

By advancing the integral photography suggested by M. G Lipmann in 1908in order to solve the above-mentioned problems, there was recentlysuggested a three-dimensional image display method using atwo-dimensional display panel such as a liquid crystal panel and apin-hole or fly-eye lens array instead of a film (for example, seeJapanese Unexamined Patent Application Publication No. 2001-275134).

In the integral photography suggested by M. G. Lipmann, a film isdisposed at a focal position of a fly-eye convex lens array and imagesof the fly-eye convex lenses are recorded on the surface of the film. Atthe time of reproducing the recorded images, the images of the fly-eyeconvex lenses recorded on the film are reproduced into a stereoscopicimage by the use of the same fly-eye convex lens array as photographingthe stereoscopic image.

In order to smoothly display a three-dimensional image with a highresolution by the use of the integral photography, it is necessary todispose different parallax images in the pinholes or lenses having asmall diameter. Here, the two-dimensional resolution of the appearancedepends upon the lens diameter of the pinhole or lens array. However,since the image information on the three-dimensional image of theappearance also relates to a density of the image formed in the depthdirection, the image information on the three-dimensional image of theappearance does not depend upon only the lens diameter of thetwo-dimensional pinhole or lens array. However, when a human being viewsa three-dimensional image and the profile of the lens is clear, decreasein resolution occurs due to recognition of the size of the profile.

In the integral photography method or the ray reproducing method, sincea convex lens array or a pinhole array made of glass or resin is used asmeans for forming a two-dimensional image including parallax images in aspace, the profile of the lens is clear, thereby decreasing thetwo-dimensional resolution. The integral photography method was embodiedusing a lenticular lens array, but the two-dimensional resolution wasdecreased due to vertical stripe profiles of the lenticular lenses.

When a three-dimensional image is displayed using the fly-eye lensarray, the lenticular lens array, or the pinhole array, crosstalk occursin the neighboring lenses or pinholes at the time of forming an imagecorresponding to display data to be displayed in a two-dimensionaldisplay unit. Accordingly, there was an attempt to reduce the crosstalkby making it difficult to form the image corresponding to the displaydata in the geometrically neighboring lenses by the use of lenses with ashort focal length. In order to embody the short focal length with anoptical system having a simple structure, it is advantageous to reducethe radius of curvature of the lenses, but it is very difficult toefficiently manufacture a lens array having a small radius of curvature.In addition, there was an attempt to optically isolate the lenses one byone by the use of a light blocking mask, but there is a problem that thereproduced three-dimensional image becomes dark and the reproduction ofthe three-dimensional image is hindered due to the profile of the lightblocking mask.

SUMMARY OF THE INVENTION

The present invention is contrived to solve the above-mentionedproblems. An object of the present invention is to provide athree-dimensional image display apparatus which can reproduce athree-dimensional image having an appearance of solidity with a simpleconfiguration.

According to an aspect of the present invention for accomplishing theabove-mentioned object, there is provided a three-dimensional imagedisplay apparatus comprising: a display unit for displayingtwo-dimensional element images having a plurality of viewing-pointimages containing information on a parallax and an image at the time ofreproducing a three-dimensional image; and a lens for forming thethree-dimensional image at a predetermined spatial position from theelement images displayed by the display unit, wherein the lens diffractsthe element images and forms the three-dimensional image by the use of adiffraction effect when the element images are emitted from the displayunit. According to this aspect of the present invention, it is possibleto enhance an appearance resolution of an image displayedthree-dimensionally and to enhance image quality, thereby reproducing athree-dimensional image having an appearance of solidity.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a configuration of a three-dimensionalimage display apparatus according to a first embodiment of the presentinvention.

FIG. 2 is a diagram illustrating a principle of an integral photographymethod.

FIG. 3 is a diagram illustrating a concept of a zone plate provided inthe three-dimensional image display apparatus according to the firstembodiment.

FIG. 4 is a diagram illustrating a configuration of a three-dimensionalimage display apparatus according to a second embodiment of the presentinvention.

FIG. 5 is a diagram illustrating a concept of a binary optical elementprovided in the three-dimensional image display apparatus according tothe second embodiment.

FIG. 6 is a diagram illustrating a configuration of a three-dimensionalimage display apparatus according to a third embodiment of the presentinvention.

FIG. 7 is a diagram illustrating a concept of a holographic lensprovided in the three-dimensional image display apparatus according tothe third embodiment.

FIG. 8 is a diagram schematically illustrating a configuration of aprojection-type three-dimensional image display apparatus according to afourth embodiment of the present invention.

FIG. 9 is a diagram illustrating a configuration of a screen of whichtransmittance can be electrically controlled.

FIG. 10A is a diagram illustrating operations of the screen of whichtransmittance can be electrically controlled and FIG. 10B is a diagramillustrating operations of the screen of which transmittance can beelectrically controlled.

FIG. 11 is a diagram schematically illustrating a configuration of aprojection-type three-dimensional image display apparatus according to afifth embodiment of the present invention.

FIG. 12 is a diagram illustrating an appearance of a projection-typethree-dimensional image display apparatus according to a sixthembodiment of the present invention.

FIG. 13 is a functional block diagram illustrating an IP image formingapparatus according to a seventh embodiment of the present invention.

FIG. 14 is a block diagram illustrating the IP image forming apparatusaccording to the seventh embodiment of the present invention relativelyto specific hardware elements.

FIG. 15 is a diagram illustrating a geometrical relation between a voxelcube and an image of the voxel cube formed on an IP image display planeaccording to the seventh embodiment of the present invention.

FIG. 16 is a flowchart schematically illustrating entire operations ofthe IP image forming apparatus according to the seventh embodiment ofthe present invention.

FIG. 17 is a flowchart illustrating an operation of obtaining an IPimage by performing a reverse ray tracing process to voxel cubes sortedin the order of decreasing a distance from a viewing point according tothe seventh embodiment of the present invention.

FIG. 18 is a diagram illustrating positional relations among a viewingline, a voxel cube, a fly-eye lens, and an IP image display planeaccording to the first embodiment of the present invention.

FIG. 19 is a flowchart illustrating an operation of obtaining an IPimage by performing a ray tracing process to voxel cubes sorted in theorder of increasing a distance from a viewing point according to theseventh embodiment of the present invention.

FIG. 20 is a functional block diagram illustrating an IP image formingapparatus according to an eighth embodiment of the present invention.

FIG. 21 is a block diagram illustrating the IP image forming apparatusaccording to the eighth embodiment of the present invention relativelyto specific hardware elements.

FIG. 22 is a flowchart schematically illustrating entire operations ofthe IP image forming apparatus according to the eighth embodiment of thepresent invention.

FIG. 23 is a flowchart illustrating an operation of obtaining an IPimage by performing a reverse ray tracing process to voxel cubes sortedin the order of decreasing a distance from a viewing point according tothe eighth embodiment of the present invention.

FIG. 24 is a flowchart illustrating an operation of obtaining an IPimage by performing a ray tracing process to voxel cubes sorted in theorder of decreasing a distance from a viewing point according to theeighth embodiment of the present invention.

FIG. 25 is a functional block diagram illustrating an IP image formingapparatus according to a ninth embodiment of the present invention.

FIG. 26 is a block diagram illustrating the IP image forming apparatusaccording to the ninth embodiment of the present invention relatively tospecific hardware elements.

FIG. 27 is a flowchart schematically illustrating entire operations ofthe IP image forming apparatus according to the ninth embodiment of thepresent invention.

FIG. 28 is a flowchart illustrating an operation of obtaining an IPimage by performing a reverse ray tracing process to the voxel cubessorted in the order of increasing a distance from a viewing pointaccording to the ninth embodiment of the present invention.

FIG. 29 is a diagram illustrating a positional relation between thevoxel cubes and pixels on the IP image display plane.

FIG. 30 is a functional block diagram illustrating a three-dimensionalimage reproducing apparatus according to a tenth embodiment of thepresent invention.

FIG. 31 is a block diagram illustrating the three-dimensional imagereproducing apparatus according to the tenth embodiment of the presentinvention relatively to specific hardware elements.

FIG. 32 is a diagram illustrating positional relations among an IPimage, an object, and a three-dimensional image when the object is infront of a lens and on the back of the lens according to the tenthembodiment of the present invention.

FIG. 33 is a flowchart schematically illustrating entire operations ofthe three-dimensional image reproducing apparatus according to the tenthembodiment of the present invention.

FIG. 34 is a flowchart illustrating a process of rendering the object infront of the lens in the three-dimensional image reproducing apparatusaccording to the tenth embodiment of the present invention.

FIG. 35 is a flowchart illustrating a process of rendering the object onthe back of the lens in the three-dimensional image reproducingapparatus according to the tenth embodiment of the present invention.

FIG. 36 is a diagram illustrating a positional relation between two IPimage display planes and the lens according to the tenth embodiment ofthe present invention.

FIG. 37 is a functional block diagram illustrating a three-dimensionalimage reproducing apparatus according to an eleventh embodiment of thepresent invention.

FIG. 38 is a block diagram illustrating the three-dimensional imagereproducing apparatus according to the eleventh embodiment of thepresent invention relatively to specific hardware elements.

FIG. 39 is a flowchart illustrating a process of rendering the object onthe back of the lens in the three-dimensional image reproducingapparatus according to the eleventh embodiment of the present invention.

FIG. 40 is a flowchart illustrating a process of rendering the object onthe back of the lens in the three-dimensional image reproducingapparatus according to the eleventh embodiment of the present invention.

FIG. 41 is a flowchart illustrating a process of rendering the object infront of the lens in the three-dimensional image reproducing apparatusaccording to the eleventh embodiment of the present invention.

FIG. 42 is a diagram illustrating a positional relation between two IPimage display planes and the lens according to the eleventh embodimentof the present invention.

FIG. 43 is a diagram illustrating a positional relation between thevoxel cubes and pixels on the IP image display plane.

FIG. 44 is a diagram illustrating a positional relation between an IPimage of an object in front of the lens and a three-dimensional image ofthe object.

FIG. 45 is a diagram illustrating a positional relation between an IPimage of an object on the back of the lens and a three-dimensional imageof the object.

FIG. 46 is a diagram illustrating a positional relation between IPimages and three-dimensional images when only one sheet of IP imagedisplay plane exists.

FIG. 47 is an enlarged diagram illustrating an IP image of an object infront of a lens as seen through the lens.

FIG. 48 is an enlarged diagram illustrating an IP image of an object onthe back of the lens as seen through the lens.

DETAILED DESCRIPTION

Hereinafter, embodiments of the present invention will be described withreference to FIGS. 1 to 48.

First Embodiment

A three-dimensional image display apparatus according to a firstembodiment of the present invention will be now described with referenceto FIGS. 1 to 3. FIG. 1 is a diagram illustrating a configuration of thethree-dimensional image display apparatus according to the firstembodiment of the invention. In FIG. 1, a reference numeral 1 denotes ahorizontal and vertical parallax image displayed by thethree-dimensional image display apparatus, a reference numeral 2 denotesa display device for displaying the horizontal and vertical parallaximage 1, a reference numeral 3 denotes a lens for forming parallaximages of the horizontal and vertical parallax image 1 in a space, and areference numeral 3 denotes a lens array in which the lenses 3 arecollected in a plane shape.

The three-dimensional image display apparatus including the horizontaland vertical parallax images 1, the display device 2, and the lens array4 having a plurality of lenses 3 displays a three-dimensional image bythe use of the integral photography. The horizontal and verticalparallax image 1 is an element image serving as a reproduction elementand is displayed as a three-dimensional image through the reproductionby the three-dimensional image display apparatus. The horizontal andvertical parallax image 1 includes image information on color, luster,texture, shape, and the like of an object at the time of reproduction asa three-dimensional image or information on color, luster, texture,shape, and the like of an object which are varied depending upon theposition of a viewing line to an image.

The three-dimensional image display apparatus projects the horizontaland vertical parallax image 1 to an image forming point 5 in space. Thehorizontal and vertical parallax image 1 projected to the image formingpoint 5 by the three-dimensional image display apparatus is incident onan observer's eyes 6 and becomes a three-dimensional image including aplurality of horizontal and vertical parallaxes. Specifically, therespective parallax images of the horizontal and vertical parallax image1 displayed on the display device 2 of the three-dimensional imagedisplay apparatus are formed at the image forming point in space by theuse of a diffraction effect by the lens 3.

The horizontal and vertical parallax image 1 is an image serving as abase of forming the three-dimensional solid image 7. The horizontal andvertical parallax image 1 is recorded in advance by disposing a film ata focal position of the lenses 3 and recording an image on the surfaceof the film every lens 3. The display device 2 is a display device fordisplaying the horizontal and vertical parallax image 1 and includes,for example, a liquid crystal panel or the like.

The integral photography method is now described in detail. FIG. 2 is adiagram illustrating a principle of the integral photography method. Theintegral photography is a method of reproducing a three dimensionalimage, which was suggested by M. G Lipmann in 1908.

In the integral photography method, a film is disposed at a focalposition of a convex lens array having a fly eye shape and an image isrecorded on the surface of the film every convex lens. At the time ofreproducing the recorded images, the image recorded on the film for eachconvex lens having a fly eye shape is reproduced as a three-dimensionalsolid image by the use of the same convex lens array having a fly eyeshape as photographing the image.

As shown in FIG. 2, when reproduction element images 8 are displayed onthe display device 9 for displaying the reproduction element images 8 tocorrespond to the convex lenses 11 of the convex lens array 10 having afly eye shape, the reproduction element images 8 are formed at an imageforming point 12 corresponding to a pixel position on the surface of theoriginal image through the respective convex lenses 11. Accordingly, byallowing rays 13 created from the image forming point 12 to be actuallyincident on the observer's eyes 14, a three-dimensional solid image 15providing a feeling of solidity to the observer is reproduced.

Since the image forming point 12 exists in space, the observer canstably view a three-dimensional solid image having an appearance ofsolidity even when an angle at which the three-dimensional image displayapparatus is disposed or an angle at which the observer views thethree-dimensional solid image 15 is changed, or even when the positionsof the eyes are moved. In the first embodiment, a zone plate which is akind of lens employing a diffraction effect is used instead of theconvex lens array 10 having a fly eye lens.

The zone plate used in the three-dimensional image display apparatusaccording to the first embodiment is now described. FIG. 3 is a diagramillustrating a concept of the zone plate of the three-dimensional imagedisplay apparatus according to the first embodiment.

In FIG. 3, a reference numeral 16 denotes an entire shape of the zoneplate. The zone plate 16 is an optical device which can form an image bybending the traveling direction of rays by the use of diffraction oflight and collecting the rays on a point.

In the zone plate 16, transparent rings and opaque rings which formconcentric circles on a substrate are alternately disposed. The diameterof each ring is proportional to a square root of a natural numbercounted from the center. For example, the radius r of a ringconstituting the zone plate 16 can be calculated from r=√nλf, where f isa focal length, λ is a wavelength of incident light, and n is a naturalnumber.

In the zone plate 16, shapes of rings obtained through predeterminedcalculation are printed in advance on a photograph or a photosensitiveresin. When parallel light 17 is incident in the direction perpendicularto the printed surface of the zone plate 16, the incident light 17 isdiffracted by diffraction portions 18 of the zone plate 16 and outgoinglight 19 of which the traveling direction is bent is emitted therefrom.The outgoing light 19 forms an image on a focal length 21 in an opticalaxis 20 passing through the center of the ring patterns of the zoneplate 16. In this way, the zone plate 16 serves as a lens.

Details of the zone plate 16 are now described with reference to FIG. 1.The diameter of the lens 3 employing the zone plate 16 is about 1 mm andthe resolution of a liquid crystal panel as the display device 2 fordisplaying the horizontal and vertical parallax image 1 is about 200dpi.

In this case, the reproduction element image 8 of the lens employing onezone plate 16 is a 10×10 pixel image and a smooth three-dimensionalsolid image of which the number of horizontal and vertical parallaxes is10 eye-shots can be reproduced. In order to obtain a smooththree-dimensional solid image, the reproduction element images 8 with aresolution of, for example, 200 to 700 dpi can be reproduced by the useof the lens array 4 employing the zone plates 16 with a diameter of 1 mmor less. The means (display device 2) for displaying the horizontal andvertical parallax image 1 is not limited to the liquid crystal panel,but may employ a plasma display panel, an organic EL panel, or the like.

According to the first embodiment, since the reproduction element images8 are reproduced by the use of the lens array 4 employing the zone plate16, an observer can little recognize the profiles of the lenses, therebyeasily reducing the crosstalk by the use of lenses with a short focallength. Accordingly, the three-dimensional image display apparatus canenhance the appearance resolution of a three-dimensional image with asimple configuration and thus can enhance the image quality.Accordingly, the three-dimensional image display apparatus can reproducean image having an appearance of solidity.

Second Embodiment

Next, a projection-type three-dimensional image display apparatusaccording to a second embodiment of the present invention will bedescribed with reference to FIGS. 4 and 5. FIG. 4 is a diagramillustrating a configuration of the three-dimensional image displayapparatus according to the second embodiment of the present invention.In FIG. 4, a reference numeral 22 denotes a light source, a referencenumeral 23 denotes light emitted from the light source 22, a referencenumeral 24 denotes a condenser lens, a reference numeral 25 denotes aprojection device, a reference numeral 26 denotes a projection lens, areference numeral 27 denotes a prism, a reference numeral 28 denotes animage forming screen, a reference numeral 29 denotes a binary opticalelement employing a diffraction effect, and a reference numeral 30denotes a three-dimensional solid image.

The light source 22 is a device for emitting the light 23 and includes,for example, a white light emitting diode (LED). The condenser lens 24collects the light 23 emitted from the light source 22.

The projection device 25 is projection means for controlling the shapeof a projection image and receives the light collected by the condenserlens 24. The projection device 25 modulates the received light into aprojection image (a modulated pattern formed on a reflecting surface ofthe projection device 25) and adds image information thereto. Theprojection device 25 includes, for example, transmissive liquid crystal.

The projection lens 26 receives the light modulated by the projectiondevice 25 and outputs the received light to the prism 27. The prism 27changes a projection angle of the light output from the projection lens26 and projects the light to the image forming screen 28.

The image forming screen 28 forms a projection image by the use of thelight projected from the prism 27. The image forming screen 28 includes,for example, a hologram diffusion plate. The binary optical element 29is means for controlling the light to form an image in a space by theuse of the diffraction effect of light and serves to form athree-dimensional solid image 30 in the space on the basis of theprojection image formed on the image forming screen 28.

Next, a procedure in which the three-dimensional image display apparatusforms the projection image will be described. In FIG. 4, the light 23emitted from the light source 22 is collected by the condenser lens 24and is input to the projection device 25. The projection device 25modulates the light input from the condenser lens 24 into the projectionimage and adds the image information thereto. Accordingly, theprojection device 25 controls the shape of the projection image.

The light modulated by the projection device 25 passes through theprojection lens 26 and a projection angle of the light is changed by theprism 27. The light of which the projection angle is changed by theprism 27 is projected to the image forming screen 28, thereby formingthe projection image.

Since the projection image formed on the image forming screen 28 is aray-traced image by the integral photography method, it is possible toform the three-dimensional solid image 30 in a space through the use ofthe binary optical elements 29 as the means for forming an image in aspace by the use of the diffraction effect of light.

In the second embodiment, the binary optical element is used as themeans for forming the element images having the image information in aspace by using the diffraction effect of light. The binary opticalelements 29 are two-dimensionally arranged in a plane parallel to theplane in which the image forming screen 28 forms the projection image,and serve as a lens array.

The binary optical element used in the three-dimensional image displayapparatus according to the second embodiment is now described in detail.FIG. 5 is a diagram illustrating a concept of the binary optical elementdisposed in the three-dimensional image display apparatus according tothe second embodiment.

In FIG. 5, a reference numeral 31 denotes the whole shape of a binaryoptical element. The binary optical element 31 is formed byapproximating a diffraction optical element having a blaze-shapedsection such as a Fresnel lens to a step shape of an element sectionalportion 32.

In the binary optical element 31 having the element sectional portion32, a diffraction grating having a micro shape is formed on the surfaceof a transparent substrate. The binary optical element 31 forms an imageby bending the traveling direction of light to collect the light on apoint by the use of the diffraction effect of light.

When parallel light 33 is incident on the surface of the binary opticalelement 31 in the direction perpendicular to the backside of the elementsectional portion 32, the incident light 33 is diffracted by diffractionportions 34 of the binary optical element 31 and outgoing light 35 ofwhich the traveling direction is bent is output from the binary opticalelement 31.

The outgoing light 35 output from the binary optical element 31 forms animage at a focal point 37 in an optical axis 36 passing through thecenter of pattern of the binary optical element 31. Accordingly, thebinary optical element 31 serves as a lens.

The diameter of the binary optical element 31 according to the presentinvention is about 1.5 mm and the resolution of the projection imageprojected to and formed on the image forming screen 28 is about 200 dpi.In this case, the reproduction element image of each binary opticalelement 31 is a 10×10 pixel image and a smooth three-dimensional solidimage of which the number of horizontal and vertical parallaxes is 10eye-shots can be reproduced. In order to obtain the smooththree-dimensional solid image, the reproduction element image with aresolution of, for example, 200 to 700 dpi can be effectively reproducedby the use of the binary optical element 31 with a diameter of 1.5 mm orless.

Although the three-dimensional image display apparatus according to thesecond embodiment includes the white LED as the light source 22, thelight source 22 is not limited to the white LED, but may include an LEDof various colors, an organic EL device, a halogen lamp, or the like.

In addition, although the three-dimensional image display apparatusaccording to the second embodiment includes the hologram diffusion plateas the image forming screen 28, the image forming screen 28 is notlimited to the hologram diffusion plate, but may include an emboss typediffusion plate or a hologram screen.

According to the second embodiment, since the three-dimensional image isformed by the use of the binary optical element 29 employing the binaryoptical element 31, an observer can little recognize the profiles of thelenses, thereby easily reducing the crosstalk by the use of lenses witha short focal length. Accordingly, the three-dimensional image displayapparatus can enhance the appearance resolution of the three-dimensionalimage with a simple configuration and thus can enhance the imagequality. As a result, the three-dimensional image display apparatus canreproduce an image having an appearance of solidity.

Third Embodiment

A three-dimensional image display apparatus according to a thirdembodiment of the present invention will be now described with referenceto FIGS. 6 and 7. FIG. 6 is a diagram illustrating a configuration ofthe three-dimensional image display apparatus according to the thirdembodiment of the present invention. In FIG. 6, a reference numeral 1denotes a horizontal and vertical parallax image displayed by thethree-dimensional image display apparatus, a reference numeral 2 denotesa display device for displaying the horizontal and vertical parallaximage 1, a reference numeral 6 denotes a lens for forming parallaximages of the horizontal and vertical parallax image 1 in a space, and areference numeral 44 denotes a lens array in which the lenses 6 arecollected in a plane shape.

The three-dimensional image display apparatus including the horizontaland vertical parallax images 1, the display device 2, and the lens array44 having a plurality of lenses 6 displays a three-dimensional image bythe use of the integral photography method.

The three-dimensional image display apparatus projects the horizontaland vertical parallax image 1 to an image forming point 5 in a space.The horizontal and vertical parallax image 1 projected to the imageforming point 5 by the three-dimensional image display apparatus isincident on an observer's eyes 43 and becomes a three-dimensional solidimage 7 including a plurality of horizontal and vertical parallaxes.Specifically, the respective parallax images of the horizontal andvertical parallax image 1 displayed on the display device 2 of thethree-dimensional image display apparatus are formed at the imageforming point 5 in a space by the use of the diffraction effect of lightby the lens 6.

The horizontal and vertical parallax image 1 is an image serving as abase of forming the three-dimensional solid image 7. The horizontal andvertical parallax image 1 is recorded in advance by disposing a film ata focal position of the lenses 6 and recording an image on the surfaceof the film every lens 6 by the use of the diffraction effect of light.

The display device 2 is a device for displaying the horizontal andvertical parallax image 1 and includes, for example, a liquid crystalpanel or the like. In the third embodiment, the lens array 44 includes,for example, holographic lenses.

The holographic lens used in the three-dimensional image displayapparatus according to the third embodiment is now described. FIG. 7 isa diagram illustrating a concept of the holographic lens used in thethree-dimensional image display apparatus according to the thirdembodiment.

In FIG. 7, a reference numeral 45 denotes the whole shape of theholographic lens. The holographic lens 45 is a diffraction opticalelement in which a hologram is formed in photosensitive gelatin such asphotopolymer or silver halide gelatin.

The holographic lens 45 is constructed on a substrate 46 made ofphotopolymer or silver halide gelatin. By creating a spherical wave froma focal position of the substrate 46 by the use of a point light source47 and inputting parallel light 49 to the substrate 46 in the directionopposite to the direction of the incident light for reproducing an imageto expose the substrate 46, a diffraction grating is formed due tovariation in refraction index therein. The diffraction grating can bendthe traveling direction of light by the use of the diffraction effect oflight, thereby collecting the light on a point to form an image.

When predetermined parallel light 50 is incident on the holographic lens45, the incident light 50 is diffracted by the diffraction grating ofthe holographic lens 45 and outgoing light 51 of which the travelingdirection is bent is output from the holographic lens 45.

The outgoing light 51 from the holographic lens 45 forms an image at afocal point 53 in the optical axis 52 passing through the center of theholographic lens 45. Accordingly, the holographic lens 45 serves as alens.

Details of the holographic lens 45 are now described with reference toFIG. 6. The diameter of the holographic lens 45 according to the presentinvention is about 1 mm and the resolution of a liquid crystal panel asthe display device 2 for displaying the horizontal and vertical parallaximage 1 is about 200 dpi. In this case, the horizontal and verticalparallax image of each holographic lens 45 is a 10×10 pixel image and asmooth three-dimensional solid image 7 of which the number of horizontaland vertical parallaxes is 10 eye-shots can be reproduced. In order toobtain a smooth three-dimensional solid image 7, the horizontal andvertical parallax image 1 with a resolution of, for example, 200 to 700dpi can be reproduced by the use of the holographic lens 45 with adiameter of 1 mm or less.

Although the three-dimensional image display apparatus according to thethird embodiment includes the holographic lens 45, the present inventionis not limited to the holographic lens 45, but the three-dimensionalimage display apparatus may include a kinoform.

According to the third embodiment, since the horizontal and verticalparallax image 1 is reproduced by the use of the lens array 44 employingthe holographic lens 45, an observer can little recognize the profilesof the lenses, thereby easily reducing the crosstalk by the use oflenses with a short focal length. Accordingly, the three-dimensionalimage display apparatus can enhance the appearance resolution of athree-dimensional image with a simple configuration and thus can enhancethe image quality. As a result, the three-dimensional image displayapparatus can reproduce an image having an appearance of solidity.

In fourth to eleventh embodiments to be described below, technologiesfor more conveniently displaying a three-dimensional image in thethree-dimensional image display apparatus and the projection-typethree-dimensional image display apparatus according to the first tothird embodiments of the present invention are described. Thethree-dimensional image display apparatus and the projection-typethree-dimensional image display apparatus according to the presentinvention can be used as a three-dimensional display apparatus and asolid image reproducing apparatus suitable in the fields of imagetechnology, amusement, entertainment, Internet, information, multimedia,communication, advertisement and promotion, medicine, art, education,design support, simulation, virtual reality, and the like.

Hereinafter, exemplary embodiments of the projection-typethree-dimensional image display apparatus according to the presentinvention will be described in detail with reference to the drawings.The present invention is not limited to the following technologies, butmay be modified in various forms without departing from the gist of thepresent invention.

Fourth Embodiment

A projection-type three-dimensional image display apparatus according tothe present invention will be described with reference to FIGS. 8 to 11.FIG. 8 is a diagram schematically illustrating a configuration of theprojection-type three-dimensional image display apparatus according to afourth embodiment of the present invention. As shown in FIG. 8, theprojection-type three-dimensional image display apparatus according tothe fourth embodiment includes a light source 101, a polarized beamsplitter 103 disposed in the traveling direction of light emitted fromthe light source 101, a projection device 104 disposed on the backsideof the polarized beam splitter 103, a projection lens 105 disposed infront of the polarized beam splitter 103, an image forming screen 106disposed in front of the projection lens 105, a convex lens array 107disposed on the front main surface of the image forming screen 106, anda light sensor 120 connected to the image forming screen 106. In FIG. 8,the side on which a three-dimensional image 8 is formed is a front sideand the opposite side is a back side.

In the projection-type three-dimensional image display apparatusaccording to the fourth embodiment, the light 102 emitted from the lightsource 101 in FIG. 8 is incident on the polarized beam splitter 103, andonly S-wave light is reflected by the boundary of the polarized beamsplitter 3 and is incident on the projection device 104 as projectionmeans for controlling the shape of a projection image. An example of theprojection device 104 can include D-ILA (Direct Drive Image LightAmplifier) made by Victor Company of Japan Limited.

An example of the projection means can include a transmissive liquidcrystal display device, a reflective liquid crystal display device, adigital mirror device, an organic EL element array, and a spatial lightmodulator (SLM). Specifically, a reflective liquid crystal projectiondevice with vertical alignment can be used as the reflective liquidcrystal display device. By using the projection device having excellentfront brightness, uniformity of a screen, and resolution as describedabove, a plurality of element images having parallax information can beexcellently projected to a two-dimensional plane, thereby embodying athree-dimensional image reproducing apparatus with high quality.

The S-wave light incident on the projection device 104 is modulated inaccordance with projection image information by the projection device104 and image information is added thereto. At this time, sincenon-modulated S-wave light is reflected and is returned to the lightsource 101 through the opposite path of the incident path, the light isnot emitted but displays black. On the other hand, the modulated S-wavelight is converted into light including a P-wave component dependingupon the degree of modulation and is projected to the image formingscreen 106 through the polarized beam splitter 103 and the projectionlens 105, thereby forming a projection image. Here, in the fourthembodiment, since the formed projection image is a ray-traced image bythe integral photography method, the projection image can form thethree-dimensional image 108 in an opposite space of the projection lensthrough the convex lens array 107.

In the projection-type three-dimensional image display apparatusaccording to the fourth embodiment, a pinhole array or a hologram lensarray can be used instead of the convex lens array 111 having a fly eyeshape. In this case, it is possible to satisfactorily form athree-dimensional reproduction image having a vertical parallax and ahorizontal parallax in a space in front of the front surface of theimage forming screen 106, thereby embodying the three-dimensional imagedisplay apparatus with high quality.

In the projection-type three-dimensional image display apparatusaccording to the fourth embodiment shown in FIG. 8, a halogen lamp isused as the light source 101. In addition to the halogen lamp, LEDs ofvarious colors or an organic EL device may be used as the light source101. Specifically, the white LED is most suitably used instead of thehalogen lamp. By using them as the light source 101, it is possible toproject projection light with high brightness, thereby displaying athree-dimensional image with high image quality.

In the projection-type three-dimensional image display apparatusaccording to the fourth embodiment, a liquid crystal screen is employedas the image forming screen 106 of which transmittance can beelectrically controlled. Accordingly, since it is possible tosatisfactorily form the projection image and to easily andsatisfactorily control the light transmittance of the screen, athree-dimensional image reproducing apparatus with high quality can beembodied.

FIG. 9 is a diagram illustrating a configuration of the screen of whichtransmittance can be electrically controlled. In the liquid crystalscreen, a liquid crystal layer 117 is interposed between two sheets oftransparent resin substrates 119 a and 119 b of which the front surfacesare coated with transparent electrodes 118 a and 118 b made of ITO orthe like. An applied voltage controller 121 for controlling a voltageapplied between the transparent electrode 118 a and the transparentelectrode 118 b is connected to the transparent electrodes 118 a and 118b. The light sensor 120 for detecting the brightness of an environmentin which the projection-type three-dimensional image display apparatusis placed is connected to the applied voltage controller 121.

When the element images as viewing-point images having parallaxinformation are projected onto the screen to display a three-dimensionalimage, the light sensor 120 of the projection-type three-dimensionalimage display apparatus detects the brightness of the environment inwhich the projection-type three-dimensional image display apparatus isplaced. The applied voltage controller 121 of the projection-typethree-dimensional image display apparatus controls the transmittance ofthe screen to change the applied voltage 122 on the basis of thebrightness of the external environment detected by the light sensor 120and controls the contrast of the element images as the viewing-pointimages having the parallax information to obtain the optimum contrast.Accordingly, since the contrast of the element images having theparallax information can be adjusted to the optimum status, it ispossible to embody a three-dimensional image reproducing apparatus withhigh quality.

As shown in FIG. 10A, when no voltage is applied between the transparentelectrode 118 a and the transparent electrode 118 b, the incident light124 is scattered by liquid crystal polymers (random liquid crystalpolymers 123) randomly existing in the liquid crystal layer 117 and isnot transmitted through the image forming screen 106. On the other hand,when a predetermined voltage is applied between the transparentelectrode 118 a and the transparent electrode 118 b, as shown in FIG.10B, the liquid crystal polymers in the liquid crystal layer 117 arealigned into aligned liquid crystal polymers 125. Accordingly, thetransmittance is enhanced to make the liquid crystal layer 117transparent. As a result, the incident light 126 becomes a transmittedlight passing through the image forming screen 106. The transmittance iscontrolled in the range of voltage applied to the liquid crystal layer117 through the transparent electrodes 118 a and 118 b. For example,when the element images of the viewing-point image having the parallaxinformation is not projected, the existence of the image forming screen106 is made to be invisible by enhancing the transmittance of the imageforming screen 106 to the maximum transmission status to make the imageforming screen 106 transparent.

A compound film of polymers and liquid crystal can be used as the imageforming screen 106 of which the transmittance can be electricallycontrolled. The compound film of polymers and liquid crystal serves tocontrol transmission and scattering by impregnating a porous member withthe liquid crystal, changing the refraction index of the liquid crystaldepending upon the application of an electric field, and adjusting thematching and mismatching with the refraction index of the porous member.This method is very useful because it is possible to overcomedisadvantages of the conventional liquid crystal in principle withoutusing a polarizing film.

Since the transparent screen at the time of application of a voltageserves as a screen at the time of application of no voltage, it ispossible to control the transmission status by the use of the voltageand to keep constant the contrast varying depending on the environmentalbrightness to stabilize the image quality. The screen can bemanufactured by the use of nematic liquid crystal which is capsulatedout of polyvinyl alcohol, liquid crystal including various latexes, or amethod of dispersing and hardening liquid crystal in epoxy resin. Amethod of manufacturing PLCC using photo-curable vinyl compound is alsoknown. This method has excellent productivity.

In the projection-type three-dimensional image display apparatusaccording to the fourth embodiment, the diameter of the convex lens 112is about 1.5 mm and the resolution of the projection image projected toand formed on the image forming screen 106 is about 200 dpi. In thiscase, the reproduction element image 109 of each lens is a 10×10 pixelimage and a smooth three-dimensional solid image of which the number ofhorizontal and vertical parallaxes is 10 eye-shots can be reproduced.When the resolution of the formed projection image is less than or equalto 200 dpi, it is difficult to obtain a smooth solid reproduced image.In order to obtain a three-dimensional solid image as smooth aspossible, the reproduction element image 109 with a resolution of 200 to700 dpi can be effectively reproduced by the use of a lens with adiameter of 1.5 mm or less.

In the projection-type three-dimensional image display apparatusaccording to the fourth embodiment, since the convex lens array 107 isintegrally formed on the front main surface of the image forming screen106, it is possible to embody a projection-type three-dimensional imagedisplay apparatus with a small size, a saved space, and low price.

According to the fourth embodiment described above, it is possible toembody a projection-type three-dimensional image display apparatus whichcan be recognized as an empty space when no image is projected, whichcan provide a realistic sensation like a solid space when athree-dimensional image is projected, and which can enhance theresolution and image quality of a three-dimensional image displayed in aspace, thereby enhancing the appearance of solidity.

In addition, by providing means for changing the transmittance of thescreen in response to the environmental brightness, it is possible toprevent deterioration in resolution and quality of a three-dimensionalimage displayed in a space and thus to provide a feeling of solidity.Accordingly, it is possible to embody a projection-typethree-dimensional image display apparatus capable of providing arealistic sensation like a solid space. As a result, it is possible toprovide a projection-type three-dimensional image display apparatuscapable of accomplishing quality such as high resolution, high imagequality, and high feeling of solidity, which could not be accomplishedby the convention liquid crystal panel or plasma display panel, with asimple structure and low cost by the use of a simple optical systemincluding the spatial light modulator and the image forming screen ofwhich the transmittance can be electrically controlled.

Fifth Embodiment

FIG. 11 is a diagram schematically illustrating a configuration of aprojection-type three-dimensional image display apparatus according to afifth embodiment of the present invention. As shown in FIG. 11, theprojection-type three-dimensional image display apparatus according tothe fifth embodiment includes a light source 101, a condenser lens 127disposed in front of the light source 101, a projection device 104disposed in front of the condenser lens 127, a projection lens 105disposed in front of the projection device 104, a prism 128 disposed infront of the projection lens 105, an image forming screen 106 disposedin front of the prism 128 to form a predetermined angel about the prism128, a convex lens array 107 disposed on the front main surface of theimage forming screen 106, and a light sensor 120 connected to the imageforming screen 106. In FIG. 11, the side on which a three-dimensionalimage 108 is formed is a front side and the opposite side is a backside.

In the projection-type three-dimensional image display apparatusaccording to the fifth embodiment, the light 102 emitted from the lightsource 101 in FIG. 11 is collected by the condenser lens 127 and isincident on the projection device 104 as the projection means forcontrolling the shape of a projection image. An example of theprojection device 104 can include a transmissive liquid crystal panel.

The light incident on the projection device 104 is modulated inaccordance with projection image information by the projection device104 and image information is added thereto. The light modulated by theprojection device 104 passes through the projection lens 105, and aprojection angle of the light is changed by the prism 128 and isprojected to the image forming screen 106, thereby forming a projectionimage. Here, in the projection-type three-dimensional image displayapparatus according to the fifth embodiment, since the formed projectionimage is a ray-traced image formed by the integral photography method,the projection image can form a three-dimensional image 108 in a spacethrough the convex lens array 107. In addition, in the projection-typethree-dimensional image display apparatus according to the fifthembodiment, since a projection plane is changed through the prism 128,it is possible to enhance the degree of freedom in design of theprojection-type three-dimensional image display apparatus.

In the projection-type three-dimensional image display apparatusaccording to the fifth embodiment shown in FIG. 11, a white LED is usedas the light source 101. In addition to the white LED, LEDs of variouscolors, an organic EL device, or a halogen lamp may be used as the lightsource 101. In addition, in the projection-type three-dimensional imagedisplay apparatus according to the fifth embodiment shown in FIG. 11, aliquid crystal screen is employed as the image forming screen 106 ofwhich transmittance can be electrically controlled.

In the fifth embodiment, similarly to the fourth embodiment, when theelement images of the viewing-point image having the parallaxinformation is not projected, the existence of the image forming screen106 is made to be invisible by enhancing the transmittance of the imageforming screen 106 to the maximum transmission status to make the imageforming screen transparent. When the element images of the viewing-pointimage having the parallax information is projected to form athree-dimensional solid image, the contrast of the element images of theviewing-point image having the parallax information is made to theoptimum status by controlling the transmittance of the image formingscreen 106 with change of the applied voltage depending upon theenvironmental brightness. As described with reference to the fourthembodiment, the transmittance can be controlled by the use of the lightsensor 120, or may be controlled by the use of an additional manualvariable resistor.

The diameter of the convex lens 112 according to the present inventionis about 1.5 mm similarly to the fourth embodiment and the resolution ofthe projection image projected to and formed on the image forming screen106 is about 200 dpi. In this case, the reproduction element image 109of each lens is a 10×10 pixel image and a smooth three-dimensional solidimage of which the number of horizontal and vertical parallaxes is 10eye-shots can be reproduced. When the resolution of the formedprojection image is less than or equal to 200 dpi, it is difficult toobtain a smooth reproduced solid image. Although a trade-off againstdata processing time, in order to obtain a three-dimensional solid imageas smooth as possible, the reproduction element image 109 with aresolution of 200 to 700 dpi can be effectively reproduced by the use ofa lens with a diameter of 1.5 mm or less.

As a result, similarly to the fourth embodiment, it is possible toembody a projection-type three-dimensional image display apparatus whichcan be recognized as an empty space when no image is projected, whichcan provide a realistic sensation like a solid space when athree-dimensional image is projected, and which can enhance theresolution and image quality of a three-dimensional image displayed in aspace, thereby enhancing the appearance of solidity.

Sixth Embodiment

A three-dimensional image communication terminal according to a sixthembodiment of the present invention includes the projection-typethree-dimensional image reproduction apparatus according to the secondto fifth embodiments of the present invention built therein.

In FIG. 12, a reference numeral 301 denotes a case of thethree-dimensional image communication terminal. In the case 301, forexample, three cameras 302, 303, and 304 as an input unit for inputtinga three-dimensional image can be mounted on a display unit 306.Directional image information as viewing a three-dimensional image invarious directions can be obtained by the use of three cameras 302, 303,and 304.

CCD cameras or CMOS cameras may be used as the above-mentioned cameras,which have preferably the number of pixels of about 1,000,000 to2,000,000. In order to express metallic luster or material texture of anobject such as glass, cloth, leather, and plant as the three-dimensionalobject, the amount of information with the number of pixels less than1,000,000 is too small and the amount of information with the number ofpixels greater than or equal to 1,000,000 is sufficient. The number ofpixels less than or equal to 2,000,000 can prevent the amount of data tobe transmitted by a communication terminal from increasing too large.

Of course, when it is not necessary for a communication terminal totransmit three-dimensional image information or when it is possible totransmit a large amount of data due to a high-rate transmission path orthe like, it is not necessary to limit the number of pixels to 2,000,000pixels or less.

The directional image information is two-dimensional bit map informationincluding parallax information in the vertical direction and thehorizontal direction of the photographed three-dimensional object. Thetwo-dimensional bit map information is obtained by processing thedirectional images in three directions photographed by the three camerasand interpolating images between the three images.

By carrying out display of an image by the use of the integralphotography method described in the embodiments of the present inventionusing the two-dimensional bit map information, it is possible to obtaina solid image of the photographed object. The installation positions ofcameras and the number of cameras can be arbitrarily selected, but it ispreferable that the number of cameras is three or more and the camerasare disposed around the display unit 306 to be distributed horizontallyand vertically.

This is because the horizontal parallax information and the verticalparallax information can be added to the photographed image by disposingthe cameras around the display unit 306 to be distributed vertically andhorizontally.

When the number of cameras is three, it is preferable that two cameras302 are disposed at both horizontal sides in the upper portion of thedisplay unit 306 and the other camera 303 is disposed at the center inthe lower portion of the display unit 306 below the two cameras.

In this arrangement, the amount of horizontal parallax information issufficient and the amount of vertical parallax information is slightlysmall.

However, since human eyes are sensitive to the horizontal informationbut not sensitive to the vertical information, the information on athree-dimensional object can be more efficiently delivered with a smallamount of information.

A reference numeral 305 denotes a three-dimensional image projectionunit. The three-dimensional image projection unit 305 projects thetwo-dimensional bit map information including the obtained informationon the three-dimensional object to the display unit 306.

The methods according to the second to fifth embodiments are used as aprinciple for the projection. Accordingly, the three-dimensional imageprojection unit 305 and the display unit 306 are based on the principlesaccording to the second to fifth embodiments.

Therefore, in the display unit 306, the light projected from thethree-dimensional image projection unit 305 as the image forming screenis formed as a projection image. The display unit 306 includes, forexample, a hologram diffusion plate. The binary optical elements 29 areattached to the display unit 306 and are means for controlling light toform an image in a space by the use of the diffraction effect of light.The binary optical elements serve to form the projection image formed inthe display unit 306 as a three-dimensional solid image 310 in a space.

In addition, the display unit 306 employs a liquid crystal screen so asto electrically control the transmittance thereof. When the elementimages of the viewing-point image having the parallax information arenot projected, the existence of the display unit 306 is made to beinvisible by enhancing the transmittance of the display unit 306 to themaximum transmission status to make the display unit transparent. Whenthe element images of the viewing-point image having the parallaxinformation is projected to form a three-dimensional solid image, thecontrast of the element images of the viewing-point image having theparallax information is made to the optimum status by controlling thetransmittance of the display unit 306 with change of the applied voltagedepending upon the environmental brightness. As described with referenceto the fourth embodiment, the transmittance can be controlled by the useof the light sensor 120, or may be controlled by the use of anadditional manual variable resistor.

A reference numeral 307 denotes a manipulation unit. A user can startcommunication with a counter party or display or photograph athree-dimensional object by manipulating the manipulation unit 307. Areference numeral 308 is a scanner/printer. The scanner/printer 308 candisplay the three-dimensional object in the display unit 306 by readingthe three-dimensional image information (such as the two-dimensional bitmap information including the parallax information) and can serve as aconventional facsimile to print image information from the counterparty.

A reference numeral 309 denotes a disk drive. By mounting a disk havingthe three-dimensional image information on the disk drive 309 andmanipulating the manipulation unit 307 for display, it is possible tothree-dimensionally display a solid object in the display unit 306.

A reference numeral 310 denotes an image of the three-dimensional objectdisplayed in the display unit 306. The displayed image of thethree-dimensional object may result from the three-dimensional imageinformation transmitted through the communication with the counterparty, a three-dimensional image photographed by the cameras 302, 303,and 304, the three-dimensional image information read by thescanner/printer 308 as described above, or the three-dimensional imageinformation output from the disk having the three-dimensional imageinformation mounted on the disk drive 309.

Now, other embodiments of the present invention will be described.

In the following embodiments, the axis direction perpendicular to thefly-eye lens is defined as a Z axis direction or a viewing-linedirection, the side apart from an observer with respect to apredetermined object (for example, the fly-eye lens) is defined as aback side or an opposite side, and the side close to the observer isdefined as a front side. However, the directions in the apparatus arenot limited to the above-mentioned directions.

Seventh Embodiment

FIG. 13 is a functional block diagram illustrating an IP image formingapparatus according to a seventh embodiment of the present invention. InFIG. 13, a reference numeral 1101 denotes three-dimensional data inputmeans, a reference numeral 1104 denotes a three-dimensional data memoryas three-dimensional data memory means, a reference numeral 1105 denotesa parameter memory, a reference numeral 1106 denotes a critical distancecalculator, a reference numeral 1108 denotes a reverse ray tracingengine, a reference numeral 1109 denotes a ray tracing engine, areference numeral 1110 denotes an IP image memory as IP image memorymeans, a reference numeral 1111 denotes an IP image display plane, areference numeral 1112 denotes a fly-eye lens, and a reference numeral1113 denotes a controller.

The three-dimensional data input means 1101 receives three-dimensionaldata of a voxel cube, that is, X, Y, and Z coordinates and colorinformation of the voxel cube, from an external device or storagemedium. The three-dimensional data memory 1104 temporarily stores thethree-dimensional data, that is, the X, Y, and Z coordinates and thecolor information of the voxel cube.

The parameter memory 1105 stores parameters necessary for rendering thethree-dimensional data. Examples of the parameters are as follows.

-   -   Diameter of the fly-eye lens 1112    -   Distance between the main point of the fly-eye lens 1112 and the        IP image display plane 1111    -   Geometrical arrangement of the fly-eye lens 1112    -   Density of the voxel cube (length of a side of the voxel cube)    -   Pixel size of an IP image (pixel size)

The critical distance calculator 1106 calculates Z coordinate values(hereinafter, referred to as a critical distance), in which the size ofthe voxel cube projected to the IP image display plane 1111 is equal tothe pixel size on the IP image display plane 1111, on the basis of the Xor Y coordinates of the voxel cube with reference to the parametermemory 1105.

The reverse ray tracing engine 1108 virtually traces a ray passingthrough the main point of the fly-eye lens from the voxel cube, obtainsthe pixel position where the ray intersects the IP image display plane1111 through operation, and checks that the pixel on the IP imagedisplay plane 1111 corresponding to the voxel cube is a “pixel coatedwith the color of the voxel cube.”

The ray tracing engine 1109 virtually traces rays passing through themain point of the fly-eye lens from the pixels constituting the IP imagedisplay plane, checks whether the rays intersect all the voxel cubes byoperation, and checks that the pixel on the IP image display planecorresponding to the voxel cube is a “pixel coated with the color of thevoxel cube” when the rays intersect all the voxel cubes. The IP imagememory 1110 stores the rendering result. The IP image display plane 1111displays an image on the basis of the data stored in the IP imagememory. The controller 1113 controls all the units.

FIG. 14 is a block diagram illustrating the IP image forming apparatusaccording to the seventh embodiment of the present invention relativelyto specific hardware components. In FIG. 14, the three-dimensional datainput means 1101 specifically includes a three-dimensional scanner. Thethree-dimensional data memory 1104 includes RAM. The parameter memory1105 includes ROM. The IP image memory 1110 includes V-RAM. The IP imagedisplay plane 1111 includes an LCS (liquid crystal monitor). Thecritical distance calculator 1106, the reverse ray tracing engine 1108,the ray tracing engine 1109, and the controller 1113 are constructed sothat a central processing unit (CPU) 1301 executes a program stored inthe parameter memory 1105 while referring to a variety of data stored inthe parameter memory 1105 or referring to or changing data stored in thethree-dimensional data memory 1104.

Here, the geometrical positional relation between the voxel cubes andthe fly-eye lens 1112 is equal to the positional relation shown in FIG.15, which illustrates the geometrical relation between a voxel cube andan image of the voxel cube formed on the IP image display planeaccording to the seventh embodiment of the present invention. At thistime, the size d of the IP image corresponding to the voxel cube is asfollows:d=(X+Δ)×s/(Z−Δ)−(X+Δ)×s/(Z+Δ)  (Expression 1)where a length of one side of the voxel cube=2Δ, a diameter of the lens1112=2r, a distance in the Z direction from the main point of the lens1112 to the center of the voxel cube=Z, a distance in the X directionfrom the main point of the lens 1112 to the center of the voxel cube=X,and a distance between the main point of the lens 1112 and the IP imagedisplay plane=s.

The following expression is obtained from Expression 1:d=2×s×Δ×(X+Z)/(Z2−Δ2)  (Expression 2)Therefore, when an image size (pixel size) on the IP image display planeis ε, it can be seen that the voxel cube satisfying ε≧d is determined bythe coordinates of X and Z.

The following expression is obtained from Expression 2 using d=ε andZ≧0.Z={s×Δ+(s2×Δ2+d2×Δ2+2×d×s×Δ×X)0.5}/d  (Expression 3)Here, when the value of X is determined, it can be seen that the valueof Z is obtained. Since the values of X coordinate of the voxel cubesare discrete, it can be seen that Z (X)=critical distance (X) satisfyingε≧d can be tabled by tabling the value of Z for each value of X. Byconsidering that Expression 2 is a decreasing function in Z≧0 andcomparing the critical distance of a voxel cube obtained from the tablewith the coordinate values of Z, it can be seen whether the size of thecorresponding IP image is greater than the pixel size. That is, when theZ coordinate value is greater than the critical distance, the size ofthe IP image is smaller than the pixel size.

Although the two dimensions of X and Z have been described here, theimaging size in the Y direction can be determined in the threedimensions by the same calculation. Here, the description thereof isomitted.

By considering that the lens 1112 captures the voxel cube at the minimumincident angle when d is greatest, the following expression isestablished.r=(X+Δ)×s/(Z−Δ)  (Expression 4)By erasing X from Expression 1 and Expression 4, d is obtained asfollows.d(max)=2×Δ×(s+r)/(Z+Δ)  (Expression 5)

Therefore, when the Z coordinate value of the voxel cube satisfies thefollowing conditional expression:ε≧d(max),it can be guaranteed the IP images of the neighboring voxel cubes arenot discontinuous. Therefore, the following expression is obtained fromExpression 5 using ε=d(max):Z≧ε×Δ/(Δ×s+Δ×r−ε)=maximum critical distanceAccordingly, even when the voxel cubes are captured at the minimumincident angle, it can be guaranteed that the IP images are continuous.Therefore, when the Z coordinate value is greater than the maximumcritical distance regardless of the coordinates of X (or Y), thecorresponding IP image is greater than the pixel size ε (converses arenot established). Since it cannot be said that the voxel cube is locatedto satisfy the minimum incident angle with respect to the lens 1112, thesize of the IP image may be greater than ε even when the Z coordinatevalue is less than the maximum critical distance. For example, in thevoxel cube located right above the main point of the lens 1112, evenwhen the Z coordinate value is greater than the critical distance, thesize of the corresponding image may be greater than ε.

Operations of the IP image forming apparatus will be described on thebasis of the above description. Entire operations are firstschematically described with reference to the flowchart shown in FIG. 16and then details of a rendering process are described with reference toFIGS. 17 and 19.

(Step S501)

The controller 1113 acquires three-dimensional data of an object throughthe three-dimensional data input means 1101. The input three-dimensionaldata includes X, Y, and Z coordinates of the voxel cubes into which theobject is divided with a predetermined resolution and colors of thevoxel cubes. The controller 1113 receives the three-dimensional data andthe resolution of the object and writes the received three-dimensionaldata and the resolution to the three-dimensional data memory 1104.

(Step S502)

Next, the controller 1113 writes the resolution of the voxel cubes, thatis, a “length of a side of a voxel cube (=2Δ)”, a “predetermineddistance between the main point of the fly-eye lens 1112 and the IPimage display plane 1111 (=s)”, a “diameter of the fly-eye lens 1112(=2r)”, and a “size of a pixel of the IP image display plane (=ε)” tothe parameter memory 1105 with reference to the three-dimensional datamemory 1104.

(Step S503)

The controller 1113 performs a sorting process in the descending order(the order of decreasing a distance from the observer) by using the Zcoordinate values as a sort key with reference to the three-dimensionaldata memory 1104 and writes again the sorted data to thethree-dimensional data memory 1104.

(Step S504)

The controller 1113 requests the critical distance calculator 1106 tocalculate the maximum critical distance. Then, the critical distancecalculator 1106 calculates the maximum critical distance from theabove-mentioned expressions with reference to the parameter memory 1105and additionally writes the calculated maximum critical distance to theparameter memory 1105.

(Step S505)

The controller 1113 divides the data stored in the three-dimensionaldata memory 1104 into three groups on the basis of the critical distancecalculated in step S504 with reference to the parameter memory 1105: thedata are divided into a group of voxel cubes (hereinafter, referred toas a first group) in which the Z coordinate value on the back side ofthe fly-eye lens 1112 as seen from the observer is greater than themaximum critical distance; a group of voxel cubes (hereinafter, referredto as a second group) in which the absolute value of the Z coordinatevalue in the vicinity of the fly-eye lens 1112 is smaller than themaximum critical distance; and a group of voxel cubes (hereinafter,referred to as a third group) in which the absolute value of the Zcoordinate value on the front side of the fly-eye lens 1112 is greaterthan the maximum critical distance. Here, the back side of the fly-eyelens 1112 is plus in the Z coordinate. On the other hand, step S505constitutes determination means for determining whether the coordinatevalue on a normal axis (Z axis) of the voxel cube is greater than themaximum critical distance. In addition, step S505 constitutesdetermination means for determining whether the IP image correspondingto the voxel cube is greater than the pixel on the IP image displayplane.

That is, in the normal axis perpendicular to the fly-eye lens 1112 inwhich the opposite side of the viewing point is plus, when a positionapart by a first maximum critical distance toward the opposite side ofthe viewing point from the fly-eye lens 1112 is assumed as a firstboundary position and a position apart by a second maximum criticaldistance toward the viewing point from the fly-eye lens 1112 is assumedas a second boundary position, the first group has normal axiscoordinate values greater than the first boundary position, the secondgroup has normal coordinate values less than or equal to the firstboundary position and greater than the second boundary position, and thethird group has normal coordinate values less than or equal to thesecond boundary position. The data indicating the boundaries between thegroups are additionally written to the three-dimensional data memory1104.

(Step S506)

The controller 1113 requests the reverse ray tracing engine 1108 toperform a rendering process to the voxel cubes belonging to the firstgroup and being stored in the three-dimensional data memory 1104.Details of the rendering process (reverse ray tracing process) will bedescribed later. The rendering result is written to the IP image memory1110.

(Step S507)

The controller 1113 requests the ray tracing engine 1109 to perform arendering process to the voxel cubes belonging to the second group andbeing stored in the three-dimensional data memory 1104. Details of therendering process (ray tracing process) will be described later. Therendering result is written to the IP image memory 1110.

(Step S508)

The controller 1113 requests the reverse ray tracing engine 1108 toperform a rendering process to the voxel cubes belonging to the thirdgroup and being stored in the three-dimensional data memory 1104.Details of the rendering process will be described later. The renderingresult is written to the IP image memory 1110.

In this way; by displaying contents written to the IP image memory 1110on the IP image display plane 1111 and observing the displayed imagethrough the fly-eye lens 1112, the same object as the object inputthrough the three-dimensional data input means 1101 is obtained as athree-dimensional image.

On the other hand, as described above, step S505 constitutes thedetermination means for determining whether the coordinate value on thenormal axis (Z axis) of the voxel cube is greater than the maximumcritical distance. When the coordinate value of the fly-eye lens 1112 is“0”, the absolute value of the coordinate value of the voxel cube may becompared with the maximum critical distance, whether it is located inthe front side or the back side of the fly-eye lens 1112. In this case,the data are divided into two groups. However, since the reverse raytracing process performed in step S508 is similar to that describedabove, there is no problem.

Next, details of the rendering processes are described with reference tothe flowchart shown in FIGS. 17 and 19. The reverse ray tracing processin step S506 is first described with reference to the flowchart shown inFIG. 17.

(Step S601)

The reverse ray tracing engine 1108 pays attention to the first voxelcube (VC).

(Step S602)

The reverse ray tracing engine 1108 pays attention to the first lens.

(Step S603)

The reverse ray tracing engine 1108 calculates a pixel position of theIP image from the positional relation between the lens and the voxelcube under notice. That is, when a ray is irradiated from the voxel cubeunder notice to the main point of the lens under notice, it iscalculated from the geometric positional relation which position of theIP image display plane the ray reaches (coordinates of a pixel arecalculated).

(Step S604)

The reverse ray tracing engine 1108 writes the color information of thevoxel cube to an address corresponding to the pixel position obtained instep S603, that is, the pixel position obtained from the IP image memory1110. Here, when the color information is written, the color informationof another voxel cube may be written in advance to the pixel position.FIG. 18 shows such a case. As shown in FIG. 18, when two voxel cubesexist on the line connecting the viewing point and the main point of thelens, the color information is written to the same pixel position on theIP image display plane. However, since the voxel cube data are sorted inthe order of decreasing a distance from the Z axis, the colorinformation can be overwritten to the pixel position even when the colorinformation is previously written to the pixel position. This is becauseit can be guaranteed that the voxel cube having the smaller Z coordinatevalue is always located further in front of the viewing point.Accordingly, since the color information of the voxel cube existing infront of the viewing point is always overwritten, it is possible toprevent the phenomenon that “a thing which must not be visible isvisible” when observing the three-dimensional image.

(Step S605)

The reverse ray tracing engine 1108 checks whether all the lenses areprocessed. When all the lenses are processed, step S606 is performed andotherwise step S607 is performed.

(Step S606)

The reverse ray tracing engine 1108 checks whether all the voxel cubesare processed. When all the voxel cubes are not processed, step S608 isperformed and otherwise the procedure is ended.

(Step S607)

The reverse ray tracing engine 1108 changes the lens under notice to thenext lens. Then, step S603 is performed.

(Step S608)

The reverse ray tracing engine 1108 changes the voxel cube under noticeto the next voxel cube. Then, step S602 is performed.

In this way, the process of obtaining the IP images of all the voxelcubes is ended. In the seventh embodiment, the order of changing thelens under notice is not particularly mentioned, but, for example, amethod of setting the primary scanning direction from the left-upperportion to the right-lower portion and setting the secondary scanningdirection from the left to the right may be used. A method of performingthe scanning in a zigzag manner from the upside to the downside or amethod of giving a particular order to the respective lenses on thebasis of a concept of interleave may also be used. All the methodsdescribed above do not depart from the gist of the present invention.

Next, a tracing process of step S507 is described with reference to theflowchart shown in FIG. 7.

(Step S801)

The ray tracing engine 1109 pays attention to the first pixel on the IPimage display plane 1111.

(Step S802)

The ray tracing engine 1109 checks whether the pixel under notice isalready processed. When the pixel under notice is processed, step S809is performed and otherwise step S803 is performed.

(Step S803)

The ray tracing engine 1109 pays attention to the first voxel cube.Here, the “first” voxel cube means a voxel cube located at the most backside among the voxel cubes belonging to the second group. That is, thevoxel cubes are processed from the front side to the back side.

(Step S804)

The ray tracing engine 1109 checks on the basis of the geometricalpositional relation whether the voxel cube under notice can be capturedby the use of a straight line extending from the coordinate position ofthe pixel under notice to the main point of the lens, that is, whetherthe “pixel under notice”, the “main point of a lens (a lens right abovethe pixel under notice”, and the “voxel cube under notice” exist in thesame line. This method is an “intersection check method” used in the raytracing process and a variety of methods are suggested. Here, details ofthe method are not described. When the voxel cube is captured by the useof the line passing through the main point, step S805 is performed andotherwise step S806 is performed.

(Step S805)

The ray tracing engine 1109 writes the color information of the voxelcube under notice and data indicating that “the color information iswritten” to the address corresponding to the pixel under notice in theIP image memory 1110. Here, the color information is expressed in 24bits. The IP image memory 1110 allocates 32 bits to a pixel. The mostsignificant bit indicates whether “the color information is written.”The next 24 bits indicate the color information. Data indicating “nocolor information is written” to all the pixels are initially set in theIP image memory.

(Step S806)

The ray tracing engine 1109 checks whether all the given voxel cubes areprocessed. When all the voxel cubes are not processed, step S808 isperformed and otherwise step S807 is performed.

(Step S807)

The ray tracing engine 1109 changes the voxel cube under notice to thenext voxel cube. Here, the next voxel cube means a voxel cube by onebefore in the sorted order. That is, the voxel cubes are processed fromthe front side to the back side in the Z direction. Then, step S804 isperformed.

(Step S808)

The ray tracing engine 1109 checks whether all the pixels on the IPimage display plane 1111 are processed. When it is checked that all thepixels are not processed, step S809 is performed and otherwise theprocess is ended.

(Step S809)

The ray tracing engine 1109 changes the pixel under notice to the nextpixel. Then, step S802 is performed.

As described above, in the IP image forming method according to theseventh embodiment, it is checked on the basis of the density ofconstituent elements (voxel cubes) of the three-dimensional object andthe resolution of the IP image display plane whether the image of avoxel cube has a size greater than that of the pixel on the IP imagedisplay plane. The conventional ray tracing process is performed whenthe image size of the voxel cube is greater than the pixel size and thereverse ray tracing process is performed when the image size of thevoxel cube is smaller than the pixel size, thereby forming an IP image.Accordingly, it is possible to form an IP image which fast reproduces anobject apart from the lens and which accurately reproduces an objectclose to the lens “without any sparse image sparse.”

In the IP image forming method according to the seventh embodiment, itis assumed that the fly-eye lens captures the voxel cube at the minimumincident angle and the Z coordinate value of the voxel cube of which theimage size is equal to the pixel size on the IP image display plane isthe maximum critical distance. When the Z coordinate value of the voxelcube is greater than the maximum critical distance, the IP image isformed by the use of the reverse ray tracing process and when the Zcoordinate value of the voxel cube is smaller than the maximum criticaldistance, the IP image is formed by the use of the ray tracing process.Accordingly, it cam be determined with a very small number of operationswhether the reverse ray tracing process should be used or the raytracing process should be used, thereby enhancing the processing speed.

It is preferable that the ray tracing process and the reverse raytracing process are used on the basis of the maximum critical distance.However, even by using the ray tracing process and the reverse raytracing process on the basis of a value slightly greater than themaximum critical distance, it is possible to accomplish the enhancementin processing speed. When the value slightly greater than the maximumcritical distance is used as a reference, the process using the raytracing process is added, but it is a slight increase from the point ofview of the entire processes. Accordingly, there is no practicalproblem.

In the IP image forming method according to the seventh embodiment, thevoxel cubes are sorted using the Z coordinate value as a sort key, areclassified into three groups of a group in which the Z coordinate valueis greater than the maximum critical distance, a group in which theabsolute value of the Z coordinate value is smaller than the maximumcritical distance, and a group in which the absolute value of the Zcoordinate value is greater than or equal to the maximum criticaldistance, and then are processed in the order of decreasing the distancefrom the observer. Accordingly, even when a plurality of voxel cubesprojected to the same pixel position exists, the image of the voxel cubelocated at the front side in the viewing-line direction is automaticallyoverwritten and left. Therefore, even when a plurality of voxel cubes isprojected to the same position, the process of determining which shouldbe left can be reduced, thereby enhancing the processing speed.

In the IP image forming method according to the seventh embodiment, thevoxel cubes are sorted using the Z coordinate value as a sort key andare classified into three groups of a group in which the Z coordinatevalue is greater than the maximum critical distance, a group in whichthe absolute value of the Z coordinate value is smaller than the maximumcritical distance, and a group in which the absolute value of the Zcoordinate value is greater than or equal to the maximum criticaldistance. Then, the group in which the Z coordinate value is greaterthan the maximum critical distance is first processed in the order ofdecreasing the distance from the observer, the group in which theabsolute value of the Z coordinate value is smaller than the maximumcritical distance is processed in the order of increasing the distancefrom the observer, and the group in which the absolute value of the Zcoordinate value is greater than or equal to the maximum criticaldistance is processed in the order of decreasing the distance from theobserver. Accordingly, in the ray tracing process requiring time, it ispossible to avoid the repeated performing of the ray tracing processfrom the same pixel position on the IP image display plane. As a result,since the ray tracing process requiring time is not performed thanneeded, it is possible to accomplish the enhancement in processingspeed.

In the IP image forming method according to the seventh embodiment,since the IP images of the voxel cubes of which the Z coordinates areclose to the fly-eye lens are obtained by the use of the ray tracingprocess, the images of the neighboring voxel cubes are formed to becontinuous on the IP image display plane. Accordingly, it is possible toprevent the phenomenon that “the color of the three-dimensional imageclose to the lens is faded” or “the back side is shown.”

Since the voxel cubes are processed from the front side but the pixelsto which the color information is previously written are skipped, it ispossible to overwrite the color information of the voxel cube located atthe back side to the color information of the voxel cube located at thefront side in the viewing line and to reduce the trouble of performingthe intersection check.

In the seventh embodiment, the order of changing the pixel under noticeis not particularly mentioned, but, for example, a method of setting theprimary scanning direction from the left-upper portion to theright-lower portion and setting the secondary scanning direction fromthe left to the right may be used. A method of processing a pixel rightbelow each lens and then changing the lens under notice may also beused. All the methods described above do not depart from the gist of thepresent invention.

Next, a reverse ray tracing process of step S508 is similar to theabove-mentioned reverse ray tracing process (step S506) and thusdescription thereof is omitted.

Here, an object is decomposed and modeled into cubes (cubic voxels), butthe gist of the present invention does not become different even whenspheres or rectangular solids are used instead of the cubes.

Eighth Embodiment

FIG. 20 is a functional block diagram illustrating an IP image formingapparatus according to an eighth embodiment of the present invention.FIG. 21 is a block diagram illustrating the IP image forming apparatusaccording to the eighth embodiment of the present invention relativelyto specific hardware components. As shown in FIGS. 20 and 21, the IPimage forming apparatus according to the eighth embodiment includesthree-dimensional media reading means 1102 for reading media in whichthe X, Y, and Z coordinates and the color information of the voxel cubesare recorded as the three-dimensional data input means. Thethree-dimensional media reading means 1102 specifically includes a DVDdrive as shown in FIG. 21. In addition, the IP image forming apparatusaccording to the eighth embodiment includes a critical distance memorytable 1107 for storing the relation between the critical distance andthe X or Y coordinates to correspond to each other. Other elements aresimilar to those of the IP image forming apparatus according to theseventh embodiment.

Operations of the IP image forming apparatus according to the eighthembodiment will be now described with reference to the flowcharts shownin FIGS. 22 to 24. First, the entire operations are schematicallydescribed with reference to the flowchart shown in FIG. 22 and thendetails of a rendering process are described with reference to FIGS. 23and 24.

(Step S1101)

The controller 1113 acquires three-dimensional data of an object storedin advance in media by the use of the three-dimensional data mediareading means 1102. The acquired data include X, Y, and Z coordinates ofvoxel cubes into which the object is divided with a predeterminedresolution and object colors of the voxel cubes. The controller 1113receives the three-dimensional data and the resolution of the object andwrites the received three-dimensional data and resolution to thethree-dimensional data memory 1104.

(Step S1102)

Next, the controller 1113 writes the resolution of the voxel cubes, thatis, a “length of a side of a voxel cube (=2Δ)”, a “distance between themain point of the fly-eye lens and the IP image display plane 1111(=s)”, a “diameter of the fly-eye lens (=2r)”, and a “size of a pixel ofthe IP image display plane (=ε)” to the parameter memory 1105 withreference to the three-dimensional data memory 1104.

(Step S1103)

The controller 1113 performs a sorting process in the descending order(the order of decreasing a distance from the observer) by using thevalues of Z coordinates as a sort key with reference to thethree-dimensional data memory 1104 and writes again the sorted data tothe three-dimensional data memory 1104. Here, the side apart in theviewing line direction from the observer is set to a plus side of the Zaxis.

(Step S1104)

The controller 1113 requests the critical distance calculator 1106 tocalculate the critical distance. Then, the critical distance calculator1106 calculates coordinates in which the voxel cubes can existdiscretely with reference to the parameter memory 1105, calculates themaximum critical distance corresponding to the X (or Y) coordinates fromthe above-mentioned expressions, and writes the calculation result tothe critical distance memory table 1107.

(Step S1105)

The controller 1113 pays attention to the first voxel cube withreference to the three-dimensional data memory 1104.

(Step S1106)

The controller 1113 checks whether the Z coordinate value is greaterthan the maximum critical distance on the basis of the X, Y, and Zcoordinates of the voxel cube under notice, with reference to thecritical distance memory table 1107. When the Z coordinate value isgreater than the maximum critical distance, step S1108 is performed andotherwise step S1107 is performed. Here, step S1106 constitutes thecheck means for checking whether the IP image corresponding to the voxelcube is greater than the pixel size on the IP image display plane withreference to the critical distance memory table 1107.

(Step S1107)

The controller 1113 requests for acquiring the IP image of the voxelcube under notice by the use of the reverse ray tracing process. Theresult is written to the IP image memory 1110. Details of the reverseray tracing process are described later. Then, step S1109 is performed.

(Step S1108)

The controller 1113 requests for acquiring the IP image of the voxelcube under notice by the use of the ray tracing process. The result iswritten to the IP image memory 1110. Details of the ray tracing processare described later.

(Step S1109)

The controller 1113 checks whether all the voxel cubes are processed.When all the voxel cubes are not processed, step S1110 is performed andotherwise the process is ended.

(Step S1110)

The controller 1113 changes the voxel cube under notice to the nextvoxel cube. Then, step S1106 is performed.

In this way, by displaying the contents written to the IP image memory1110 on the IP image display plane 1111 and observing the displayedcontents through the fly-eye lenses 1113, the same object as the objectread by the three-dimensional data media reading means 1102 is obtainedas a three-dimensional image.

Next, details of the rendering processes are described with reference tothe flowcharts shown in FIGS. 23 and 24. The reverse ray tracing processof step S1107 is first described with reference to the flowchart shownin FIG. 23. Here, the reverse ray tracing engine 1108 receives one voxelcube to be processed from the controller 1113.

(Step S1201)

The reverse ray tracing engine 1108 pays attention to the first lens.

(Step S1202)

The reverse ray tracing engine 1108 calculates a pixel position of theIP image from the positional relation between the lens under notice andthe given voxel cube. That is, when a ray is irradiated from the voxelcube under notice to the main point of the lens under notice, it iscalculated from the geometric positional relation which position of theIP image display plane the ray reaches (coordinates of a pixel arecalculated).

(Step S1203)

The reverse ray tracing engine 1108 writes the color information of thevoxel cube to an address corresponding to the pixel position obtained instep S1202, that is, the pixel position obtained from the IP imagememory 1110. Here, when the color information is written, the colorinformation of another voxel cube may be written in advance to the pixelposition, but the color information can be overwritten to the pixelposition. Accordingly, since the color information of the voxel cubeexisting in front of the viewing point is always overwritten, it ispossible to prevent the phenomenon that “a thing which must not bevisible is visible” when observing the three-dimensional image.

(Step S1204)

The reverse ray tracing engine 1108 checks whether all the lenses areprocessed. When all the lenses are not processed, step S1205 isperformed and otherwise the procedure is ended.

(Step S1205)

The reverse ray tracing engine 1108 changes the lens under notice to thenext lens and then performs step S1202.

In this way, the process of obtaining the IP images of the given voxelcubes is ended.

In the eighth embodiment, the order of changing the lens under notice isnot particularly mentioned, but, for example, a method of setting theprimary scanning direction from the left-upper portion to theright-lower portion and setting the secondary scanning direction fromthe left to the right may be used. A method of performing the scanningin a zigzag manner from the upside to the downside or a method of givinga particular order to the respective lenses on the basis of a concept ofinterleave may also be used. All the methods described above do notdepart from the gist of the present invention.

Next, the ray tracing process of step S1108 is described with referenceto the flowchart shown in FIG. 24. Here, the ray tracing engine 1109receives one voxel cube to be processed from the controller 1113.

(Step S1301)

The ray tracing engine 1109 pays attention to the first pixel on the IPimage display plane 1111.

(Step S1302)

The ray tracing engine 1109 checks on the basis of the geometricalpositional relation whether the voxel cube under notice can be capturedby the use of a straight line extending from the coordinate position ofthe pixel under notice to the main point of the lens, that is, whetherthe “pixel under notice”, the “main point of a lens (a lens right abovethe pixel under notice”, and the “voxel cube under notice” exist in thesame line. This method is an “intersection check method” used in the raytracing process and a variety of methods are suggested. Here, details ofthe method are not described. When the voxel cube is captured by the useof the line passing through the main point, step S1303 is performed andotherwise step S1304 is performed.

(Step S1303)

The ray tracing engine 1109 writes the color information of the voxelcube under notice to an address corresponding to the pixel under noticein the IP image memory 1110. Here, as described relatively to thereverse ray tracing process, color information may be previously writtento the pixel under notice, but by overwriting the color information tothe pixel under notice, the color information of the voxel cube close tothe viewing point always remains, thereby preventing the phenomenon that“a thing which must not be visible is visible” when reproducing thethree-dimensional image.

(Step S1304)

The ray tracing engine 1109 checks whether all the pixels on the IPimage display plane 1111 are processed. When all the pixels are notprocessed, step S1505 is performed and otherwise the procedure is ended.

(Step S1305)

The ray tracing engine 1109 changes the pixel under notice to the nextpixel and then performs step S1302.

In the IP image forming method according to the eighth embodiment, byacquiring in advance the Z coordinate value of each voxel cube of whichthe IP image is equal to the pixel in size every X or Y coordinate andstoring the Z coordinate value in the table, the table is referred towhen acquiring the IP image. When the Z coordinate value of the voxelcube is apart from the Z coordinate value stored in the table, the IPimage is obtained by the use of the reverse ray tracing process and whenthe Z coordinate value of the voxel cube is close to the Z coordinatevalue stored in the table, the IP image is obtained by the use of theray tracing process. Accordingly, it is possible to reduce the operationamount of the ray tracing process as much as possible.

Since the IP images of the voxel cubes of which the Z coordinates areclose to the fly-eye lens are obtained by the use of the ray tracingprocess, the images of the neighboring voxel cubes are formed to becontinuous on the IP image display plane. Accordingly, it is possible toprevent the phenomenon that “the color of the three-dimensional imageclose to the lens is faded” or “the back side is shown.”

In the eighth embodiment, the order of changing the pixel under noticeis not particularly mentioned, but, for example, a method of setting theprimary scanning direction from the left-upper portion to theright-lower portion and setting the secondary scanning direction fromthe left to the right may be used. A method of processing a pixel rightbelow each lens and then changing the lens under notice may also beused. All the methods described above do not depart from the gist of thepresent invention.

Ninth Embodiment

FIG. 25 is a functional block diagram illustrating an IP image formingapparatus according to a ninth embodiment of the present invention. FIG.26 is a block diagram illustrating the IP image forming apparatusaccording to the ninth embodiment of the present invention relatively tospecific hardware components. As shown in FIGS. 25 and 26, the IP imageforming apparatus according to the ninth embodiment includesthree-dimensional data communication means 1103 for exchanging the X, Y,and Z coordinates and the color information of the voxel cubes throughcommunication with a counter party as the three-dimensional data inputmeans. The three-dimensional data communication means 1102 specificallyincludes a network interface. On the other hand, the critical distancememory table 1107 included in the IP image forming apparatus accordingto the eighth embodiment is not provided in the ninth embodiment. Otherelements are similar to those of the IP image forming apparatusaccording to the eighth embodiment.

Operations of the IP image forming apparatus according to the ninthembodiment will be now described with reference to the flowcharts shownin FIGS. 27 and 28. First, the entire operations are schematicallydescribed with reference to the flowchart shown in FIG. 27 and thendetails of a rendering process are described with reference to FIG. 28.

(Step S1601)

The controller 1113 acquires three-dimensional data of an object throughthe three-dimensional data communication means 1103. The acquired datainclude X, Y, and Z coordinates of voxel cubes into which the object isdivided with a predetermined resolution and object colors of the voxelcubes. The controller 1113 receives the three-dimensional data and theresolution of the object and writes the received three-dimensional dataand resolution to the three-dimensional data memory 1104.

(Step S1602)

Next, the controller 1113 writes the resolution of the voxel cube, thatis, a “length of a side of a voxel cube (=2Δ)”, a predetermined“distance between the main point of the fly-eye lens and the IP imagedisplay plane (=s)”, a “diameter of the fly-eye lens (=2r)”, and a “sizeof a pixel of the IP image display plane (=ε)” to the parameter memory1105 with reference to the three-dimensional data memory 1104.

(Step S1603)

The controller 1113 performs a sorting process in the ascending order(the order of increasing a distance from the observer) by using thevalues of Z coordinates as a sort key with reference to thethree-dimensional data memory 1104 and writes again the sorted data tothe three-dimensional data memory 1104.

(Step S1604)

The controller 1113 requests the critical distance calculator 1106 tocalculate the maximum critical distance. Then, the critical distancecalculator 1106 calculates the maximum critical distance from theabove-mentioned expression with reference to the parameter memory 1105and additionally writes the calculated maximum critical distance to theparameter memory 1105.

(Step S1605)

The controller 1113 additionally writes data indicating boundariesbetween groups to the three-dimensional data memory 1104 in order todivide the data stored in the three-dimensional data memory 1104 intothree groups on the basis of the critical distance stored in theparameter memory 1105: a group of voxel cubes (hereinafter, referred toas a first group) in which the absolute value of the Z coordinate valueon the front side of the fly-eye lens as seen from the observer isgreater than the maximum critical distance; a group of voxel cubes(hereinafter, referred to as a second group) in which the absolute valueof the Z coordinate value in the vicinity of the fly-eye lens 1112 issmaller than the maximum critical distance; and a group of voxel cubes(hereinafter, referred to as a third group) in which the Z coordinatevalue on the back side of the fly-eye lens 1112 is greater than themaximum critical distance. Here, the back side of the fly-eye lens 1112is plus in the Z coordinate. On the other hand, step S1605 constitutesdetermination means for determining whether the coordinate value on anormal axis (Z axis) of the voxel cube is greater than the maximumcritical distance. In addition, step S1605 constitutes determinationmeans for determining whether the IP image corresponding to the voxelcube is greater than the pixel on the IP image display plane.

(Step S1606)

The controller 1113 requests the reverse ray tracing engine 1108 toperform a rendering process to the voxel cubes belonging to the firstgroup and being stored in the three-dimensional data memory 1104.Details of the rendering process (reverse ray tracing process) will bedescribed later. The rendering result is written to the IP image memory1110.

(Step S1607)

The controller 1113 requests the ray tracing engine 1109 to perform arendering process to the voxel cubes belonging to the second group andbeing stored in the three-dimensional data memory 1104. Details of therendering process (ray tracing process) will be described later. Therendering result is written to the IP image memory 1110.

(Step S1608)

The controller 1113 requests the reverse ray tracing engine 1108 toperform a rendering process to the voxel cubes belonging to the thirdgroup and being stored in the three-dimensional data memory 1104.Details of the rendering process will be described later. The renderingresult is written to the IP image memory 1110.

In this way, by displaying contents written to the IP image memory 1110on the IP image display plane 1111 and observing the displayed imagethrough the fly-eye lens 1112, the same object as the object inputthrough the three-dimensional data communication means 1103 is obtainedas a three-dimensional image.

Next, details of the rendering processes are described with reference tothe flowchart shown in FIG. 28. The reverse ray tracing process in stepS1606 is first described.

(Step S1701)

The reverse ray tracing engine 1108 pays attention to the first voxelcube (VC).

(Step S1702)

The reverse ray tracing engine 1108 pays attention to the first lens.

(Step S1703)

The reverse ray tracing engine 1108 calculates a pixel position of theIP image from the positional relation between the lens and the voxelcube under notice. That is, when a ray is irradiated from the voxel cubeunder notice to the main point of the lens under notice, it iscalculated from the geometric positional relation which position of theIP image display plane the ray reaches (coordinates of a pixel arecalculated).

(Step S1704)

The reverse ray tracing engine 1108 checks whether the color informationof the voxel cube is written to the address in the IP image memory 1110corresponding to the position calculated in step S1703. When it ischecked that the color information is written already, step S1709 isperformed and otherwise step S1705 is performed.

(Step S1705)

The reverse ray tracing engine 1108 writes “the color information of thevoxel cube under notice” and data indicating that “the color informationis written” to “the address of the IP image memory” corresponding to thepixel position calculated in step S1703. Here, data indicating that “nocolor information is written” are initially written to the IP imagememory. The color information is expressed in 24 bits. The IP imagememory 1110 allocates 32 bits to a pixel. The most significant bitindicates “whether the color information is written.” The next 24 bitsindicate the color information.

(Step S1706)

The reverse ray tracing engine 1108 checks whether all the lenses areprocessed.

When it is checked that all the lenses are processed, step S1707 isperformed and otherwise step S1709 is performed.

(Step S1707)

The reverse ray tracing engine 1108 checks whether all the voxel cubesare processed. When all the voxel cubes are not processed, step S1708 isperformed and otherwise the procedure is ended.

(Step S1708)

The reverse ray tracing engine 1108 changes the voxel cube under noticeto the next voxel cube and then performs step S1702.

(Step S1709)

The reverse ray tracing engine 1108 changes the lens under notice to thenext lens and then performs step S1703.

In this way, the process of obtaining the IP images of the given voxelcubes is ended. Here, as the voxel cube is closer to the viewing point,it is earlier processed. The pixels to which the color information ispreviously written in the IP image memory are skipped. Accordingly, thecolor information of the voxel cube closer to the viewing point is left,thereby preventing the phenomenon that “a thing which must not bevisible is visible.”

In the ninth embodiment, the order of changing the lens under notice isnot particularly mentioned, but, for example, a method of setting theprimary scanning direction from the left-upper portion to theright-lower portion and setting the secondary scanning direction fromthe left to the right may be used. A method of performing the scanningin a zigzag manner from the upside to the downside or a method of givinga particular order to the respective lenses on the basis of a concept ofinterleave may also be used. All the methods described above do notdepart from the gist of the present invention.

Next, the ray tracing process of step S1607 is similar to the process ofstep S507 according to the seventh embodiment and thus its descriptionis omitted.

Next, the reverse ray tracing process of step S11608 is similar to theprocess (step S1606) and thus description thereof is omitted.

As described above, in the IP image forming apparatus according to theninth embodiment, the voxel cubes are sorted using the Z coordinates asa sort key, are processed in the order of increasing the distance fromthe observer, and are skipped when the IP image is formed previously.Accordingly, it is possible to prevent the reverse ray tracing processfrom being performed more than needed. As a result, it is possible toenhance the processing speed.

In the following embodiments, the axis direction perpendicular to thefly-eye lens is defined as the Z axis direction or the viewing-linedirection, the side apart from an observer with respect to apredetermined object (for example, the fly-eye lens) is defined as a“back side” or an “opposite side”, and the side close to the observer isdefined as a “front side.” However, the directions in the apparatus arenot limited to the above-mentioned directions.

Tenth Embodiment

FIG. 30 is a functional block diagram illustrating a three-dimensionalimage reproducing apparatus according to a tenth embodiment of thepresent invention. In FIG. 30, a reference numeral 2101 denotesthree-dimensional data input means, a reference numeral 2102 denotes athree-dimensional data memory as the three-dimensional data memorymeans, a reference numeral 2103 denotes a parameter memory, a referencenumeral 2104 denotes a rendering engine, a reference numeral 2105denotes a first IP image memory as the IP image memory means, areference numeral 2106 denotes a second IP image memory as the IP imagememory means, a reference numeral 2107 denotes a first IP image displayplane, a reference numeral 2108 denotes a second IP image display plane,a reference numeral 2109 denotes a fly-eye lens, and a reference numeral2111 denotes a controller.

The three-dimensional data input means 2101 receives three-dimensionaldata of voxel cubes, that is, X, Y, and Z coordinates and colorinformation of the voxel cubes, from an external device or storagemedium. The three-dimensional data memory 2102 temporarily stores thethree-dimensional data, that is, the X, Y, and Z coordinates and thecolor information of the voxel cubes.

The parameter memory 2103 stores parameters necessary for rendering thethree-dimensional data. Examples of the parameters are as follows.

-   -   Diameter of the fly-eye lens 2109    -   Distance between the main point plane of the fly-eye lens 2109        and the first IP image display plane 2107    -   Distance between the main point plane of the fly-eye lens 2109        and the second IP image display plane 2108    -   Geometrical arrangement of the fly-eye lens 2109    -   Focal length of the fly-eye lens 2109    -   Coordinates of the main pint of the fly-eye lens 2109    -   Density of a voxel cube (length of a side of the voxel cube)    -   Size of a pixel of an IP image (pixel size)    -   Arrangement of pixels    -   Transmittance of the first IP image display plane 2107

The rendering engine 2104 virtually traces rays passing through the mainpoint of the fly-eye lens 2109 from the pixels constituting the IP imagedisplay plane, and checks which voxel cube the rays intersect throughcalculation. When the rays interest any voxel cube, the rendering enginedetermines that the pixel on the IP image display plane corresponding tothe voxel cube is a “pixel coated with the color of the voxel cube.”

The first IP image memory 2105 stores the rendering result to an objectlocated at the back side from the fly-eye lens 2109. On the other hand,the second IP image memory 2106 stores the rendering result to an objectlocated at the front side from the fly-eye lens 2109. The first IP imagedisplay plane 2107 displays an image on the basis of the data stored inthe first IP image memory 2105. On the other hand, the second IP imagedisplay plane 2108 displays an image on the basis of the data stored inthe second IP image memory 2106. The controller 2111 controls all theunits.

FIG. 31 is a block diagram illustrating the three-dimensional imagereproducing apparatus according to the tenth embodiment of the presentinvention relatively to specific hardware components. In FIG. 31, thethree-dimensional data input means 2101 specifically includes a DVDdrive. The three-dimensional data memory 2102 includes RAM. Theparameter memory 2103 includes ROM. The rendering engine 2104 and thecontroller 2111 are constructed so that a central processing unit (CPU)2801 executes a program stored in the parameter memory (ROM) 2103 whilereferring to data stored in the parameter memory (ROM) 2103 or referringto or changing data stored in the three-dimensional data memory (RAM)2102.

The first IP image memory 2105 specifically includes a first V-RAM. Thesecond IP image memory 2106 specifically includes a second V-RAM. Thefirst IP image display plane 2107 specifically includes a transmissiveLCD. The second IP image display plane 2108 specifically includes abacklight LCD.

Here, the fly-eye lens 2109, the first IP image display plane 2107(transmissive LCD), and the second IP image display plane 2108(backlight LCD) are geometrically arranged as shown in FIG. 32illustrating positional relations among an IP image, an object, and athree-dimensional image, relative to the object in the front of a lensand the object on the back of the lens, according to the tenthembodiment of the present invention. That is, the first IP image displayplane 2107 is disposed between the fly-eye lens 2109 and the focal pointplane of the fly-eye lens 2109 and the second IP image display plane2108 is disposed at the opposite side of the focal point plane as seenfrom the fly-eye lens 2109. That is, when the observer side of thefly-eye lens 2109 is defined as a first surface side and the oppositeside of the observer of the fly-eye lens 2109 is defined as a secondsurface side, the first IP image display plane 2107 is disposed betweenthe fly-eye lens 2109 and the focal point plane of the fly-eye lens 2109at the second surface side of the fly-eye lens 2109. The second IP imagedisplay plane 2108 is disposed on the side opposite to the fly-eye lens2109 with respect to the focal point plane of the fly-eye lens 2109 atthe second surface side of the fly-eye lens 2109.

The three-dimensional image reproducing apparatus according to the tenthembodiment can be embodied as a three-dimensional image reproducingprogram which is driven by a general-purpose computer. Accordingly, itcan be embodied with low cost.

Operations of the three-dimensional image display apparatus having theabove-mentioned configuration will be now described. First, the entireoperations are schematically described with reference to the flowchartshown in FIG. 33 and then details of a rendering process are describedwith reference to FIGS. 34 and 35.

(Step S1001)

The controller 2111 acquires three-dimensional data of an object throughthe three-dimensional data input means 2101 from, for example, anexternal device or a storage medium. The acquired three-dimensional dataspecifically include X, Y, and Z coordinates of voxel cubes into whichthe object is virtually divided with a predetermined resolution andobject colors of the voxel cubes. The controller 2111 reads thethree-dimensional data through the three-dimensional data input means2101 and writes the read three-dimensional data to the three-dimensionaldata memory 2102.

(Step S1002)

Thereafter, the controller 2111 reads the three-dimensional data fromthe three-dimensional data memory 2102, performs a sorting process inthe descending order (the order of increasing a distance from theobserver) by using the values of Z coordinates as a sort key, and writesagain the sorted data to the three-dimensional data memory 2102.

(Step S1003)

Next, the controller 2111 divides the data into a first group A of whichthe Z coordinate values are plus and a second group B of which the Zcoordinate values are minus with reference to the three-dimensional datamemory 2102. Here, since the data stored in the three-dimensional datamemory 2102 are previously sorted with the Z coordinate values, thecontroller 2111 additionally writes information indicating the boundarybetween the first group A and the second group B. In the tenthembodiment, the front side (close to the observer) in the viewing linefrom the fly-eye lens 2109 is a plus direction of the Z coordinatevalue, the back side (apart from the observer) in the viewing line fromthe fly-eye lens 2109 is a minus direction, and the Z coordinate valueof the main point plane of the fly-eye lens 2109 is “0.”

(Step S1004)

[Rendering Process for Object in Front of Lens]

Next, the controller 2111 requests the rendering engine 2104 to performthe rendering process to the voxel cubes belonging to the first group Astored in the three-dimensional data memory 2102. Details of therendering process are described later. The rendering result is writtento the second IP image memory 2106.

(Step S1005)

[Rendering Process for Object on the Back of Lens]

Next, the controller 2111 requests the rendering engine 2104 to performthe rendering process to the voxel cubes belonging to the second group Bstored in the three-dimensional data memory 2102. Details of therendering process are described later. The rendering result is writtento the first IP image memory 2105.

In this way, by displaying contents written to the first IP image memory2105 and the second IP image memory 2106 on the first IP image displayplane 2107 and the second IP image display plane 2108 and observing thedisplayed image through the fly-eye lens 2109, the same object as theobject input through the three-dimensional data input means 2101 isobtained as a three-dimensional image.

Next, details of the rendering process are described with reference tothe flowcharts shown in FIGS. 34 and 35. FIG. 34 is a flowchartillustrating an operation of [Rendering Process for Object in front ofLens] which is performed in step S1004 of FIG. 33. in the renderingprocess for the voxel cubes (group A) located in front of the fly-eyelens 2109, it is necessary to render the voxel cubes with respect to thesecond IP image display plane 2108 opposite to the main point plane soas to reproduce the three-dimensional image without vertical and lateralinversion.

(Step S1101)

The rendering engine 2104 pays attention to the first pixel on thesecond IP image display plane 2108. The position of the pixel isdetermined based on the pixel arrangement stored in the parameter memory2103. Here, it is assumed that the geometrical position of the second IPimage display plane 2108 uniquely corresponds to the address of thesecond IP image memory 2106.

(Step S1102)

The rendering engine 2104 checks with reference to the second IP imagememory 2106 whether the pixel under notice is formed previously. Whenthe pixel under notice is formed previously, step S1110 is performed andotherwise step S1104 is performed.

(Step S1104)

The rendering engine 2104 pays attention to the first voxel cube. Here,the first voxel cube means a voxel cube located at the front end of thevoxel cubes belonging to the second group A. That is, the voxel cubesare processed from the front side to the back side.

(Step S1105)

The rendering engine 2104 checks on the basis of the geometricalpositional relation whether the voxel cube under notice can be capturedby the use of a straight line extending from the coordinate position ofthe pixel under notice on the second IP image display plane 2108 to themain point of the fly-eye lens 2109, that is, whether the “pixel undernotice”, the “main point of a fly-eye lens 2109 (a fly-eye lens 2109right above the pixel under notice)”, and the “voxel cube under notice”exist in the same line. The coordinates of the main point is stored inthe parameter memory 2103. Of which fly-eye lens 2109 the pixel undernotice is associated with the main point can be checked on the basis ofthe geometrical arrangement of the fly-eye lenses 2109, the arrangementof the pixels, the pixel size, and the coordinates of the main pointsstored in the parameter memory 2103. This method is an “intersectioncheck method” used in the ray tracing process and a variety of methodsare suggested. Here, details of the method are not described. When thevoxel cube is captured by the use of the line passing through the mainpoint, step S1107 is performed and otherwise step S1108 is performed.

(Step S1107)

The rendering engine 2104 writes the color information of the voxel cubeunder notice and data indicating that “the color information is written”to the address corresponding to the pixel under notice in the second IPimage memory 2106. In the tenth embodiment, the color information isexpressed in 24 bits. The second IP image memory 2106 allocates 32 bitsto a pixel. When the color information is written, the color informationis subjected to computation so that the first IP image display plane2107 and the second IP image display plane 2108 exhibit the same colorsby considering the transmittance of the first IP image display plane2107 stored in the parameter memory 2103. Specifically, the R, G, and Bvalues of the color information of the voxel cube are multiplied by thetransmittance and then the computation result is written as the colorinformation to the second IP image memory 2106. A method of storing inadvance the transmittances of R, G, and B in the parameter memory 2103and multiplying the R, G, and B values by the transmittances,respectively, or a method of listing the values multiplied by the R, G,and B values in a table may be considered. The methods described abovedo not depart from the gist of the present invention. The mostsignificant bit indicates whether “the color information is written.”The next 24 bits indicate the color information. Data indicating “nocolor information is written” to all the pixels are initially set in thesecond IP image memory 2106.

(Step S1108)

The rendering engine 2104 checks whether all the voxel cubes belongingto the first group A are processed. When all the voxel cubes areprocessed, step S1109 is performed and otherwise step S1111 isperformed.

(Step S1109)

The rendering engine 2104 checks whether all the pixels on the second IPimage display plane 2108 are processed with reference to the second IPimage memory 2106. When all the pixels are not processed, step S1110 isperformed and when all the pixels are processed, the procedure is ended.

(Step S1110)

The rendering engine 2104 changes the pixel under notice to the “next”pixel and then performs step S1102.

(Step S111)

The rendering engine 2104 changes the voxel cube under notice to the“next” voxel cube. Here, the “next” means one next thereto in the sortedorder. That is, the voxel cubes are processed from the front side to theback side in the Z direction. Then, step S1105 is performed.

As described above, the voxel cubes of which the Z coordinate values arelocated at the front side of the fly-eye lens are processed from thefront side. At this time, since the pixels to which the colorinformation is previously written are skipped, it is possible tooverwrite the color information of the voxel cube located at the backside (that is, voxel cube to be processed subsequently) to the colorinformation of the voxel cube located at the front side in the viewingline and to reduce the trouble of performing the intersection check.

On the other hand, in the tenth embodiment, the order of changing thepixel under notice is not particularly mentioned, but, for example, amethod of setting the primary scanning direction from the left-upperportion to the right-lower portion and setting the secondary scanningdirection from the left to the right may be used. A method of processinga pixel right below each fly-eye lens 2109 and then changing the fly-eyelens 2109 under notice may also be used. All the methods described abovedo not depart from the gist of the present invention.

Next, the rendering process for the voxel cubes (group B) on the back ofthe fly-eye lens 2109 in step S1005 is described with reference to theflowchart shown in FIG. 35. As described above, it is necessary torender the voxel cubes on the back of the fly-eye lens 2109 with respectto the first IP image display plane 2107 in front of the main pointplane so as to reproduce the three-dimensional image without verticaland lateral inversion.

(Step S1201)

The rendering engine 2104 pays attention to the first pixel on the firstIP image display plane 2107. The position of the pixel is determinedbased on the pixel arrangement stored in the parameter memory 2103.Here, it is assumed that the geometrical position of the first IP imagedisplay plane 2107 uniquely corresponds to the address of the first IPimage memory 2105.

(Step S1102)

The rendering engine 2104 checks with reference to the first IP imagememory 2105 whether the pixel under notice is formed previously. Whenthe pixel under notice is formed previously, step S1209 is performed andotherwise step S1203 is performed.

(Step S1203)

The rendering engine 2104 calculates a position where the extension ofthe straight line connecting the pixel position of the pixel undernotice on the first IP image display plane 2107 in step S1202 to theposition of the main point of the fly-eye lens 2109 right above thepixel intersects the second IP image display plane 2108 and checks withreference to the second IP image memory 2106 whether the pixel is formedpreviously at the calculated position. FIG. 36 illustrates a positionalrelation among the pixel on the first IP image display plane 2107, thepixel on the second IP image display plane 2108, and the fly-eye lens2109. When the pixel is formed previously, step S2109 is performed andotherwise step S2104 is performed.

(Step S1204)

The rendering engine 2104 pays attention to the first voxel cube. Here,the first voxel cube means a voxel cube located at the front end of thevoxel cubes belonging to the second group B. That is, the voxel cubesare processed from the front side to the back side.

(Step S1205)

The rendering engine 2104 checks on the basis of the geometricalpositional relation whether the voxel cube under notice can be capturedby the use of a straight line extending from the coordinate position ofthe pixel under notice on the first IP image display plane 2107 to themain point of the fly-eye lens 2109, that is, whether the “pixel undernotice”, the “main point of a fly-eye lens 2109 (a fly-eye lens 2109right above the pixel under notice)”, and the “voxel cube under notice”exist in the same line. The coordinates of the main point is stored inthe parameter memory 2103. Of which fly-eye lens 2109 the pixel undernotice is associated with the main point can be checked on the basis ofthe geometrical arrangement of the fly-eye lenses 2109, the arrangementof the pixels, the pixel size, and the coordinates of the main pointsstored in the parameter memory 2103. This method is an “intersectioncheck method” used in the ray tracing process and a variety of methodsare suggested. Here, details of the method are not described. When thevoxel cube is captured by the use of the line passing through the mainpoint, step S1206 is performed and otherwise step S1207 is performed.

(Step S1206)

The rendering engine 2104 writes the color information of the voxel cubeunder notice and data indicating that “the color information is written”to the address corresponding to the pixel under notice in the first IPimage memory 2105. Here, the color information is expressed in 24 bits.The first IP image memory 2105 allocates 32 bits to a pixel. The mostsignificant bit indicates whether “the color information is written.”The next 24 bits indicate the color information. Data indicating “nocolor information is written” to all the pixels are initially set in thefirst IP image memory 2105.

(Step S1207)

The rendering engine 2104 checks whether all the voxel cubes belongingto group B are processed. When all the voxel cubes are processed, stepS1208 is performed and otherwise step S1210 is performed.

(Step S1208)

The rendering engine 2104 checks whether all the pixels on the first IPimage display plane 2107 are processed with reference to the first IPimage memory 2105. When all the pixels are not processed, step S1209 isperformed and otherwise the procedure is ended.

(Step S1209)

The rendering engine 2104 changes the pixel under notice to the nextpixel and then performs step S1202.

(Step S1210)

The rendering engine 2104 changes the voxel cube under notice to thenext voxel cube. Here, the next means one next thereto in the sortedorder. That is, the voxel cubes are processed from the front side to theback side in the Z direction. Then, step S1205 is performed.

In the three-dimensional image reproducing method according to the tenthembodiment, the first IP image display plane 2107 which can make thespecified pixel transparent is disposed between the fly-eye lens 2109and the focal point plane and the second IP image display plane 2108 isdisposed at the opposite side of the focal point plane with respect tothe fly-eye lens 2109. Accordingly, the object in front of the fly-eyelens 2109 is displayed on the second IP image display plane 2108, theobject on the back of the fly-eye lens 2109 is displayed on the first IPimage display plane 2107, and the pixel where the straight lineconnecting the pixel on the second IP image display plane 2108 and thefly-eye lens 2109 intersects the first IP image display plane 2107 ismade transparent. As a result, since the object in front of the fly-eyelens 2109 and the object on the back of the fly-eye lens form an imageat the same position with the same size as the original object andwithout vertical and lateral inversion, it is possible to obtain athree-dimensionally reproduced image with a high resolution.

In the three-dimensional image reproducing method according to the tenthembodiment, when an IP image is first formed on the first IP imagedisplay plane 2107 and then an IP image is formed on the second IP imagedisplay plane 2108, the pixel at the position where the straight lineconnecting the pixel position in which the IP image is formed to themain point of the fly-eye lens 2109 intersects the first IP imagedisplay plane 2107 is made transparent. Accordingly, since the pixel onthe second IP image display plane 2108 can be necessarily seen throughthe fly-eye lens 2109 and the transparent pixel on the first IP imagedisplay plane 2107, the pixel on the first IP image display plane 2107for obtaining the three-dimensional image of the object on the back ofthe fly-eye lens 2109 does not cover the pixel on the second IP imagedisplay plane 2108 for obtaining the three-dimensional image of theobject in front of the fly-eye lens 2109, thereby reproducing thethree-dimensional image with right arrangement toward the back side ofthe viewing line.

In addition, in the three-dimensional image reproducing method accordingto the tenth embodiment, the data obtained by dividing an object into aplurality of voxel cubes are sorted with the Z coordinate values of thevoxel cubes in the descending order (here, it is assumed that the Z axisis perpendicular to the fly-eye lens 2109, the origin is set on the mainpoint plane of the fly-eye lens 2109, and the observer side is definedas plus) and the IP images corresponding to the voxel cubes are obtainedin that order. When an IP image exists on the first and second IP imagedisplay planes 2107 and 2108, the process is omitted. As a result, whenthe voxel cube at the front side as seen from the observer is firstprocessed and then a plurality of voxel cubes is mapped onto the samepixel position on the first and second IP image display planes 2107 and2108, data of the front side in the viewing line are always left.Accordingly, it is possible to prevent the phenomenon that the objectlocated at the back side in the viewing line is seen at the front sidein the viewing line. In addition, the pixels in which the IP images arepreviously formed on the first and second IP image display planes 2107and 2108 can be skipped, it is possible to reduce the operations,thereby enhancing the processing speed.

That is, in the tenth embodiment, when the IP image is formed on thefirst IP image display plane located at the front side in the viewingline, the IP image formed previously on the second IP image displayplane 2108 located at the back side in the viewing line is not covered.Accordingly, when a three-dimensional image is reproduced, the object onthe back of the fly-eye lens 2109 does not cover the object in front ofthe fly-eye lens 2109. In addition, since the three-dimensional image ofthe object in front of the fly-eye lens 2109 and the three-dimensionalimage of the object on the back of the fly-eye lens 2109 are formed atthe original positions without vertical and lateral inversion, it ispossible to obtain a three-dimensional image with a high resolution.Since the pixels on the second IP image display plane 2108 are formed inconsideration of the transmittance of the first IP image display plane2107, the observer recognizes that the color of the first IP imagedisplay plane 2107 is equal to the color of the second IP image displayplane 2108, thereby obtaining a realistic three-dimensional image.

Here, an object is divided into cubes (voxel cubes) for modeling, butthe gist of the present invention is not changed even by the use ofspheres or rectangular parallelepipeds.

Eleventh Embodiment

FIG. 37 is a functional block diagram illustrating a three-dimensionalimage reproducing apparatus according to an eleventh embodiment of thepresent invention. FIG. 38 is a block diagram illustrating thethree-dimensional image reproducing apparatus according to the eleventhembodiment of the present invention relatively to specific hardwarecomponents. As shown in FIGS. 37 and 38, the three-dimensional imagereproducing apparatus according to the eleventh embodiment includes acontrast adjustment unit 2110 for adjusting a contrast provided in thesecond IP image display plane 2107. Other elements are similar to thoseof the three-dimensional image reproducing apparatus according to thetenth embodiment.

Operations of the three-dimensional image reproducing apparatusaccording to the eleventh embodiment are described with reference to theflowcharts shown in FIGS. 39 to 41. First, the entire operations areschematically described with reference to the flowchart shown in FIG. 39and then details of the rendering process are described. In the tenthembodiment, the rendering process is first performed to the object infront of the fly-eye lens 2109 and then the rendering process isperformed to the object located at the back side. However, in theeleventh embodiment, the rendering process is first performed to theobject located at the back side and then the rendering process isperformed to the object located at the front side. In order to clearlyunderstand the structural features of the eleventh embodiment, theblocks carrying out the same operations as those of the tenth embodimentare simply described in brief and difference is concentrically describedin detail.

(Steps S1601 to S1603)

The operations of steps S1601 to S1603 according to the eleventhembodiment are similar to the operations of steps S1001 to S1003according to the tenth embodiment shown in the flowchart of FIG. 33.That is, the three-dimensional data are read and are then rearranged inthe descending order using the Z coordinate values as the sort key.Thereafter, the three-dimensional data are classified into a first groupA in which the Z coordinate values are plus and a second group B inwhich the Z coordinate values are minus.

(Step S1604)

Next, the controller 2111 requests the rendering engine 2104 to renderthe voxel cubes belonging to the first group A stored in thethree-dimensional data memory 2102. That is, the rendering process isperformed to the object on the back of the fly-eye lens 2109. Therendering result is written to the first IP image memory 2105.

(Step S1605)

Next, in the eleventh embodiment, the controller 2111 requests therendering engine 2104 to render the voxel cubes belonging to the secondgroup B stored in the three-dimensional data memory 2102. That is, therendering process is performed to the object in front of the fly-eyelens 2109. The rendering result is written to the second IP image memory2106.

Next, details of the rendering process are described with reference tothe flowcharts shown in FIGS. 40 and 41. FIG. 40 is a flowchartillustrating an operation of [Rendering Process for Object on the backof Lens] which is performed in step S1604 of FIG. 39. This operation isapproximately equal to the flowchart according to the tenth embodimentshown in FIG. 35. However, in the eleventh embodiment, since therendering process is first performed to the object on the back of thelens, such step S1203 of checking whether formation for the pixel iscompleted is not required.

Similarly to the tenth embodiment, it is necessary to render the voxelcubes on the back of the fly-eye lens 2109 with respect to the first IPimage display plane 2107 in front of the main point plane so as toreproduce the three-dimensional image without vertical and lateralinversion.

(Steps S1701 and S1702)

The operations of steps S1701 and S1702 according to the eleventhembodiment are similar to the operations of steps S1201 and S1202 of theflowchart according to the tenth embodiment shown in FIG. 35. That is,the rendering engine 2104 checks with reference to the first IP imagememory 2105 whether the formation for the pixel under notice iscompleted. When it is YES, the rendering engine then pays attention tothe next pixel in step S1709 and otherwise step S1704 is performed. Inthe eleventh embodiment, as described above, since the rendering processis first performed to the object on the back of the lens, such stepS1203 of checking whether the formation for the pixel is completed isnot required.

(Steps S1704 to S1709)

Next, operations of steps S1704 to SI 709 according to the eleventhembodiment are similar to those of steps S1204 to 1209 in the flowchartaccording to the tenth embodiment shown in FIG. 35.

In the eleventh embodiment, similarly to the tenth embodiment, the voxelcubes of which the Z coordinate values are located at the front side ofthe fly-eye lens 2109 are processed from the back side. At this time,since the pixels to which the color information is previously writtenare skipped, it is possible to overwrite the color information of thevoxel cube located at the back side (that is, voxel cube to be processedsubsequently) to the color information of the voxel cube located at thefront side in the viewing line and to reduce the trouble of performingthe intersection check.

[Rendering Process for Object in Front of Lens]

Next, the rendering process for the voxel cubes (group A) in front ofthe fly-eye lens 2109 in step S1605 is described with reference to theflowchart shown in FIG. 41. Similarly to the tenth embodiment, it isnecessary to render the voxel cubes in front of the fly-eye lens 2109with respect to the second IP image display plane 2108 on the back ofthe main point plane so as to reproduce the three-dimensional imagewithout vertical and lateral inversion.

(Step S1801 and S1802)

The operations of steps S1801 and S1802 according to the eleventhembodiment are similar to the operations of steps S1101 and S1102 of theflowchart according to the tenth embodiment shown in FIG. 34. In theeleventh embodiment, since the rendering process is first performed tothe object on the back of the fly-eye lens 2109, step S1803 is added incomparison with the flowchart shown in FIG. 34.

(Step S1803)

The rendering engine 2104 calculates the position where the straightline connecting the position of the pixel under notice in step S1802 onthe second IP image display plane 2108 to the position of the main pointof the fly-eye lens 2109 right above the pixel intersects the first IPimage display plane 2107, and then checks with reference to the first IPimage memory 2105 corresponding to the position when the formation forthe corresponding pixel is completed. FIG. 42 illustrates a positionalrelation among the pixel on the first IP image display plane 2107, thepixel on the second IP image display plane 2108, and the fly-eye lens2109. When it is YES, step S1810 is performed and otherwise step S1804is performed.

(Step S1804)

Next, the rendering engine 2104 pays attention to the first voxel cube.Here, the first voxel cube means a voxel cube located at the head amongthe voxel cubes belonging to the second group B. That is, the voxelcubes are processed from the front side to the back side.

(Step S1805)

Similarly to the first embodiment, the rendering engine 2104 checks onthe basis of the geometrical positional relation whether the voxel cubeunder notice can be captured by the use of a straight line extendingfrom the coordinate position of the pixel under notice on the second IPimage display plane 2108 to the main point of the fly-eye lens 2109,that is, whether the “pixel under notice”, the “main point of a fly-eyelens 2109 (a fly-eye lens 2109 right above the pixel under notice)”, andthe “voxel cube under notice” exist in the same line. The coordinates ofthe main point is stored in the parameter memory 2103. Of which fly-eyelens 2109 the pixel under notice is associated with the main point canbe checked on the basis of the geometrical arrangement of the fly-eyelenses 2109, the arrangement of the pixels, the pixel size, and thecoordinates of the main points stored in the parameter memory 2103. Thismethod is an “intersection check method” used in the ray tracing processand a variety of methods are suggested. Here, details of the method arenot described. When the voxel cube is captured by the use of the linepassing through the main point, step S1807 is performed and otherwisestep S1806 is performed.

(Step S1806)

The rendering engine 2104 writes the data indicating “transparent”, thatis, R=0, G=0, and B=0, to the address in the first IP image memory 2105corresponding to the pixel position calculated in step S1803. The mostsignificant bit indicates “whether the color information is written.”The next 24 bits indicate the color information. Data indicating “nocolor information is written” to all the pixels are initially set in thefirst IP image memory 2105.

(Step S1807)

The rendering engine 2104 writes the color information of the voxel cubeunder notice and data indicating that “the color information is written”to the address corresponding to the pixel under notice in the second IPimage memory 2106. Here, the color information is expressed in 24 bits.The second IP image memory 2106 allocates 32 bits to a pixel. The mostsignificant bit indicates whether “the color information is written.”The next 24 bits indicate the color information. Data indicating “nocolor information is written” to all the pixels are initially set in thesecond IP image memory 2106.

(Steps S1808 to S1811)

The operations of steps S1808 to S1811 according to the eleventhembodiment are similar to the operations of steps S1108 to S1111 of theflowchart according to the tenth embodiment shown in FIG. 34.

In the IP image forming method according to the eleventh embodiment,when an IP image is first formed on the first IP image display plane2107 and then an IP image is formed on the second IP image display plane2108, the pixel on the first IP image display plane at the positionwhere the straight line connecting the pixel position in which the IPimage is formed to the main point of the fly-eye lens 2109 intersectsthe first IP image display plane 2107 is made transparent. Accordingly,since the pixel on the second IP image display plane 2108 can benecessarily seen through the fly-eye lens 2109 and the transparent pixelon the first IP image display plane 2107, the pixel on the first IPimage display plane 2107 for obtaining the three-dimensional image ofthe object on the back of the fly-eye lens 2109 does not cover the pixelon the second IP image display plane 2108 for obtaining thethree-dimensional image of the object in front of the fly-eye lens 2109,thereby reproducing the three-dimensional image with right arrangementtoward the back side of the viewing line.

In the IP image forming method according to the eleventh embodiment, thedegree of fadeness in color of the second IP image display plane 2108 asseen through the transparent pixel on the first IP image display plane2107 is numerically expressed in consideration of the degree oftransparence of the first IP image display plane 2107, and the colordisplayed on the first IP image display plane 2107 is faded.Accordingly, the color of the first IP image display plane 2107 and thecolor of the second IP image display plane 2108 exhibit the same tone.As a result, since the object in front of the fly-eye lens 2109 and theobject on the back of the fly-eye lens are reproduced in the samedynamic range of color, it is possible to obtain a three-dimensionalimage without discomfort.

In the three-dimensional image reproducing apparatus according to theeleventh embodiment, the contrast adjustment unit 2110 is provided inthe second IP image display plane 2108. Accordingly, since the dynamicrange of color when viewing the second IP image display plane 2108through the transparent portion of the first IP image display plane 2107and the dynamic range of color of the first IP image display plane 2107can be adjusted, it is possible to reproduce a three-dimensional imagewithout discomfort.

That is, in the eleventh embodiment, when the IP image formed on thefirst IP image display plane 2107 located at the front side in theviewing line hinders the formation of the IP image on the second IPimage display plane 2108 located at the back side in the viewing line,it is possible to prevent the hindrance by replacing it with atransparent pixel. Accordingly, when a three-dimensional image isreproduced, the object on the back of the fly-eye lens 2109 does notcover the object in front of the fly-eye lens 2109. In addition, sincethe three-dimensional image of the object in front of the fly-eye lens2109 and the three-dimensional image of the object on the back of thefly-eye lens 2109 are not inverted vertically and laterally, it ispossible to obtain a three-dimensional image with a high resolution. Inaddition, by adjusting the contrast adjustment unit in consideration ofthe transmittance of the fist IP image display plane 2107, the observerrecognizes that the color of the first IP image display plane 2107 isequal to the color of the second IP image display plane 2108, therebyobtaining a realistic three-dimensional image.

In the eleventh embodiment, when the color information is written to theposition where the straight line connecting a pixel on the second IPimage display plane 2108 to the main point of the fly-eye lens 2109intersects the first IP image display plane 2107, the pixel on the firstIP image display plane 2107 is made transparent and then the colorinformation is written to the pixel on the second IP image display plane2108. However, it does not depart from the gist of the present inventioneven if the processing order is inverted.

1. A three-dimensional image display apparatus comprising: a displayunit for displaying two-dimensional element images having a plurality ofviewing-point images containing information on a parallax and an imageat the time of reproducing a three-dimensional image; and a lens forforming the three-dimensional image at a predetermined spatial positionfrom the element images displayed by the display unit, wherein the lensdiffracts the element images to form the three-dimensional image by theuse of a diffraction effect when the element images are emitted from thedisplay unit.
 2. The three-dimensional image display apparatus accordingto claim 1, wherein the lens includes a zone plate in which transparentrings and opaque rings are alternately formed in a concentric circle. 3.The three-dimensional image display apparatus according to claim 1,wherein the lens includes a binary optical element having a blaze-shapedsection.
 4. The three-dimensional image display apparatus according toclaim 3, wherein the binary optical element is a Fresnel lens.
 5. Thethree-dimensional image display apparatus according to claim 1, whereinthe lens includes a hologram lens formed by hologram.
 6. Thethree-dimensional image display apparatus according to claim 1, whereinthe lens includes a kinoform which is a phase hologram.
 7. Aprojection-type three-dimensional image display apparatus comprising:projection means for projecting a plurality of element images containingparallax information to a two-dimensional plane; a screen of which lighttransmittance can be electrically controlled and on which a projectionimage including the plurality of element images projected by theprojection means is formed; and image forming means for forming athree-dimensional image having a vertical parallax and a horizontalparallax in a space in front of the screen from the projection image. 8.The projection-type three-dimensional image display apparatus accordingto claim 7, wherein transmissive liquid crystal is used as theprojection means.
 9. The projection-type three-dimensional image displayapparatus according to claim 7, wherein reflective liquid crystal isused as the projection means.
 10. The projection-type three-dimensionalimage display apparatus according to claim 8, wherein a digital mirrordevice is used as the projection means.
 11. The projection-typethree-dimensional image display apparatus according to claim 7, whereinan organic EL element array is used as the projection means.
 12. Theprojection-type three-dimensional image display apparatus according toclaim 7, wherein a spatial light modulator is used as the projectionmeans.
 13. The projection-type three-dimensional image display apparatusaccording to claim 7, wherein a light emitting diode is used as a lightsource of the projection means.
 14. The projection-typethree-dimensional image display apparatus according to claim 12, whereina white LED is used as a light source of the projection means.
 15. Theprojection-type three-dimensional image display apparatus according toclaim 7, wherein an organic electroluminescence device is used as alight source of the projection means.
 16. The projection-typethree-dimensional image display apparatus according to claim 7, whereinthe projection means includes means for changing a projection directionof the plurality of element images having the parallax information. 17.The projection-type three-dimensional image display apparatus accordingto claim 7, wherein a liquid crystal screen is used as the screen. 18.The projection-type three-dimensional image display apparatus accordingto claim 17, wherein light transmittance of the screen is changeddepending upon brightness of an external environment.
 19. Theprojection-type three-dimensional image display apparatus according toclaim 7, wherein contrast of the element images having the parallaxinformation is adjusted by controlling the light transmittance of thescreen into the maximum transmission status when the element imageshaving the parallax information are not projected and by varying thelight transmittance of the screen in accordance with brightness of anexternal environment when the element images having the parallaxinformation are projected.
 20. The projection-type three-dimensionalimage display apparatus according to claim 7, wherein a hologram lensarray is used as the image forming means.
 21. A projection-typethree-dimensional image display apparatus comprising a projection unitfor projecting a plurality of element images having parallax informationto a two-dimensional plane, a screen capable of electrically controllinglight transmittance on which a projection image including the pluralityof element images projected by the projection unit is formed, and a lensfor forming a three-dimensional image in a space in front of the screenfrom the element images projected by the projection unit, wherein thelens diffracts the element images to form the three-dimensional image bythe use of a diffraction effect when the element images are emitted froma display unit.